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CHAPTER  I 


INTRODUCTION 


Ln  RECENT  years  the  use  of  queueing  networks  as  models  for  complex 
stochastic  systems  has  expanded  dramatically.  In  computer  modeling,  for 
example,  cueueing  netwo'ks  have  been  used  as  models  cf  data  management 
systems,  systems  with  many  peripheral  processors  and  demand  paging  sys¬ 
tems,  among  many  others.  As  the  models  become  more  detailed,  analytical 
results  become  very  difficult  to  obtain.  For  this  reason,  simulation  has  become 
a  •  increasingly  attractive  avenue  for  the  stu  f  of  these  models. 

Typi<  s'.*y,  the  goal  of  the  simulation  is  the  estimation  of  some  characteris¬ 
tic  of  the  system  under  study.  However,  determining  how  close  the  simulation's 
estimates  are  to  the  correct  value  is  a  major  problem.  A  common  approach  to 
th'.«  difficulty  is  the  use  of  confidence  intervals.  This  method  is  most  fruitful 
when  the  variance  of  the  estimate  is  rasiiy  obtained;  but,  unfortunately,  this 
is  often  a  difficult  problem. 

When  the  characteristic  under  study  is  a  function  of  the  long  run  ‘equili¬ 
brium’  behavior  of  the  system,  a  special  class  of  processes  called  regenerative 
processes  can  sometimes  be  used  to  greatly  simplify  the  problem  of  variance 
esti;.  ntion.  These  processes  periodically  'begin  from  scratch'  in  the  sense  that 
at  certain  epochs  in  time,  the  system  behaves  as  it  did  originally.  Tiii>  allows 
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each  sample  path  to  be  broken  into  independent  and  identically  distributed 
pieces,  which  makes  the  variance  calculation  straightforward.  It  is  therefore 
im  rtant  to  find  ways  to  model  queueing  networks  as  regenerative  processes. 
This  is  our  first  goal. 

One  approach  to  modeling  queueing  networks  and  other  complex  stochas¬ 
tic  systems  which  ha*  received  some  attention  in  the  literature  is  the  generalised 
semi-Markov  process  (GS.MP).  This  idea  is  an  example  of  the  supplementary 
variables  approach  to  nou-Markovian  systems  lescribed  in  Cox  and  Miller 
(1986).  This  approach  ‘supplements’  the  natural  description  of  the  system  by 
variables  which  contain  information  about  th-  past  history  of  the  system.  In 
this  way,  a  model  of  a  non-Markovi&n  ystem  can  be  made  Markovian.  For 
GSMPs  the  supplementary  variables  are  clocks  which  record  the  amount  of 
time  until  the  occurrence  of  various  events  which  could  inSuence  the  rystem. 
In  a  que.  ring  network,  for  example,  each  server  and  each  arrival  stream  would 
be  associated  with  a  clock.  By  including  these  clock  reading j  as  part  of  the 
description  of  the  system,  only  the  present  state  of  the  system  !s  required  to 
predict  future  behaviour.  This  means  that  the  new  model  is  Markovian  and 
therefore  amenable  to  a-  ilysi*  via  the  use  of  Markov  chain  theory. 

To  use  these  processes  for  v.mulatton  purposes  a  central  limit  theorem  is 
required.  Obtaining  this  result  is  the  second  goal  of  this  paper.  Our  approach 
to  this  problem  is  to  find  closely  related  r*j,rnerative  processes  on  which  to 
base  the  central  limit  theorem  for  the  process  under  study.  New  results  in 
the  theory  of  Markov  chains  on  a  general  state  space  make  it  clear  how  these 
regenerative  processes  can  be  constructed. 

A  natural  starting  point  for  obtain  mg  our  objectives  is  a  review  of  the 
theory  of  Markov  chains.  Section  2.1  contains  a  review  of  the  applicable 
limit  theorems.  The  hypotheses  for  thes-  results  have  developed  m  various 
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recurrence  conditions  and  these  are  also  reviewed.  The  structure  and  notation 
of  generalized  semi-Markov  processes  are  introduced  in  Section  2.2.  The  final 
section  of  Chapter  II  is  a  brief  summary  of  the  regenerative  approach  to 
simulation. 

The  main  results  of  this  paper  are  presented  in  two  chapters.  The  first 
section  of  Chapter  III  identifies  a  class  of  GSMPs  which  are  regenerative.  If  a 
queueing  network  occupies  a  state  where  only  one  event  can  change  the  state 
of  the  system,  (when  all  jobs  in  a  closed  network  wait  for  a  single  server,  for 
example)  the  process  beh  wes  the  same  way  each  time  it  leaves  this  state. 
This  phenomenon  can  be  observed  in  any  GSMP  that  has  a  single  event  a*»- 
sociated  with  any  state.  This  is  the  critical  feature  of  regenerative  processes. 
Section  3.2  presents  two  examples  where  regenerative  methods  are  used  in  the 
simulation  of  queueing  networks. 

For  those  GSMPs  without  a  ’single’  state,  Chapter  IV  describes  how 
results  of  Markov  chain  th-ory  can  be  applied  to  obtain  central  limit  theorems 
for  functions  of  the  equilibrium'  behavior  of  the  GSMP.  The  results  in  Section 
4.2  determine  which  GSMPs  satisfy  the  recurrence  conditions  introduced  in 
Section  2.1.  As  long  as  the  clocks  governing  the  events  of  the  GSMP  have 
bounded  positive  densities  on  the  positive  half  line,  at  least  the  weakest  forms 
of  recurrence  are  satisfied.  Section  4.3  presents  an  ergodic  theorem  for  these 
GSMPs.  A  large  class  of  these  recurrent  GSMPs  can  be  associated  with  a 
regenerative  process  by  a  further  use  of  the  supplementary  variables  tech¬ 
nique.  By  supplementing  the  GSMP  with  a  'memory'  of  a  finite  number  of 
steps,  au  iterate  of  the  transition  function  of  the  GSM?  can  be  split  into 
a  state-invariant  measure  and  a  state-dependent  remainder.  By  replacing 
this  iterate  of  the  transition  function  by  a  judicious  choice  between  the  in¬ 
variant  measure  and  the  remainder  a  regenerative  process  can  be  formed 
which  has  marginal  distributions  identical  to  the  GSMP’s.  Unfortunately,  this 
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regenerative  process  is  not  suitable  for  simulation  applications  because  the  the 
remainder  cannot  be  determined  explicitly  in  most  cases.  To  circumvent  this 
problem,  we  propose  a  new  supplementary  variables  technique  which  only 
requires  a  sequence  of  Bernoulli  random  variables.  The  development  of  this 
procedure  and  the  central  limit  theorem  for  the  new  supplemented  process  is 
discussed  in  Section  *1.4.  Estimation  of  the  central  limit  theorem's  variance 
constant  is  the  topic  oi  Section  4.5.  An  example  testing  extensions  of  the 
theory  is  presented  in  Section  4.6. 

Chapter  V  examines  the  strengths  and  weaknesses  of  this  new  technique. 

Before  beginning,  it  is  convenient  to  define  some  notatiou  which  will  be 
used  throughout  our  discussion.  The  indicator  function  la(z)  is  1  or  0  accord¬ 
ing  as  x  £  D  or  x  B.  Then  if  X  is  a  random  variable,  I/?(X)  is  also,  and 
equals  1  if  the  event  {  X  £  B  }  occurs  and  0  if  it  does  not.  We  will  use  Rn  to 
denote  the  crovs  product  of  n  copies  of  the  real  numbers.  If  n  is  omitted,  it 
will  be  assumed  to  be  1. 

All  results  with  original  proofs  will  be  labelled  theorems,  while  results 
cited  from  other  sources  will  be  called  lemmas.  The  symbol  3  will  denote 
the  conclx.iion  of  a  proof. 


CHAPTER n 


PRELIMINARIES 


2.1.  MARKOV  CHAINS  ON  A  GENERAL  STATE  SPACE 

Ln  THIS  section  notation  and  some  limiting  results  are  introduced  for 
Markov  chains  on  a  general  state  space.  These  results  are  useful  is  the  develop¬ 
ment  of  central  limit  theorems  for  GSMPs.  Most  of  these  theorems  demand 
that  the  chain  satisfy  some  kind  of  recurrence  property  and  a  review  of  t'.-.cse 
conditions  is  an  important  part  of  the  discussion.  For  a  mere  complete  dis¬ 
cussion  of  these  notions,  see  Neveu  (1984),  Orey  (1971)  a:ii  Revue  (1975). 
Throughout  this  section  (E.S)  will  be  a  measurable  space. 

(I)  DEFINITION.  A  function  P:(£7,  S)  — *  (0,  1]  is  said  to  be  a  probability 
traositiou  function  (or  kernel)  if: 

(a)  P(x,  •)  is  a  probability  measure  on  (E,8)  for  all 

(b)  P(-,Z?)  is  a  measurable  function  with  respect  to  G,  for  all  P  £  G. 
The  n-step  transition  probabilities  are  defined  by  setting  Pl(x,2i)  ^  P(x,B) 
and  P"+l(x,B)  -  fs P"(x.  dy)P(y.B). 

In  the  usual  fashion,  let  £7°°  ■«  E  X  E  X  £7  X  •  •  •  and  G°°  «■  6  X  6  X  •  •  • . 
For  w  €  E,  let  A',(o>)  be  the  ith  coordinate  of  w.  For  any  probability  measure 
i/  on  (£7,  8),  there  is  a  probability  measure  P  on  (£7°°,  6°”)  satisfying  for  any 
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n  >  0  ami  S  the  following  relation: 


P*(A'o  6  Bo,  Xi  £  Bit . . . ,  X„  E  Bn) 

—  [  u{dx o)  /  P(xo,  dx\)-  •  •  f  P(i„_ i  dxn). 

J  Bq  JBi  J 13* 

The  properties  of  the  measure  P„  ensure  that  conditional  probabilities  can  be 
constructed  which  satisfy 

P„(Xn  €  B  |  Xo  -  *o,  Xi  -  . . Xk  -  **)  -  P"~k(*k.i?),  (2) 

for  n  >  0,  x  G  E,  k  =»  0,  l . n. 

Equation  2  is  a  version  of  the  Markov  property,  and  when  it  is  satisfied 
n  >  0  }  is  said  to  be  a  Markov  chain  on  state  space  E  with  initial 
distribution  u  and  stationary  transition  probability  fuuction  P. 

The  notion  of  recurrence  captures  the  idea  of  infinitely  many  visits  by 
the  Markov  chain  to  a  state  or  a  collection  of  states.  In  a  countable  state 
space  Markov  chain,  a  state  is  called  recurrent  if,  starting  in  state  i,  the  chain 
returns  to  state  i  with  probability  1.  If,  in  addition,  all  tue  states  communicate, 
the  chain  is  said  to  be  recurrent.  If,  however,  the  state  space  of  the  Marker 
chain  is  not  countable,  it  may  not  happen  that  a  particular  state  is  visited 
infinitely  often;  a  more  general  concept  is  therefore  needed.  One  natural  way 
to  generalise  the  idea  is  to  require  that  'significant'  sets  are  visited  'frequently 
enough.’  To  study  these  conditions  a  measure  closely  related  to  P  must  first 
be  introduced. 

(3)  DEFINITION.  A  o-finite  measure  t  on  (E,6)  i3  invariant  for  the 
transition  probability  function  P  if,  for  all  x  G  E  and  A  £  6 

*{A)  ”  j^P[x,A)Adx). 
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The  Grst  general  notion  of  recurrence  is  a  local  condition  due  to  Harris 
(1956)  which  has  recently  been  reformulated  by  Athreya  and  Ney  (1978)  and 
Nummelin  (1978). 

(4)  DEFINITION,  (a)  The  chain  X  is  said  to  be  ^recurrent  (or  Harris 
recurrent)  if  there  exists  a  positive  o-finite,  invariant  measure  y?  on  E  such 
that  y?(A)>0  implies 

a* 

P(£  M*m)  -  -r«=)  -  1- 

ff%mm  1 

(b)  A  Markov  chain  X  on  (E,S)  is  [A,B,  <p,  X,  n)-recurrent  if  there  exists 
sets  A  and  B  £  6,  a  probability  measure  f  on  B,  a  Quite  number  X  >  0,  and 
a  Gnite  integer  n  such  that 

(i)  P«(A*  £  A  for  some  k  >  1)  —  1  for  all  and 

(ii)  P,(X„  £  C)  >  Xy>(C)  for  all  x  £  A  and  C  C  B. 

Lf  eitiier  (a)  or  (b)  is  satisGed  X  will  be  said  to  be  recurrent.  If  in  part 
(b)  A—*B,  then  the  chain  will  be  said  to  be  (A,  >p,  n)-recurrent.  (A,yj,  X,  n)- 
recurrence  is  in  fact  no  less  general  than  (A,f?,<p,  X,  n)-recurr.rnce.  A  proof  of 
this  fact  end  of  the  equivalence  of  parts  (a)  and  (b)  can  be  found  in  Athreya 
i-.ad  Ney  (1977,  1978).  Several  limit  results  for  these  chains  are  of  interest. 

(5)  LEMMA.  For  every  recurrent  Markov  chain  on  (E,E)  with  transi¬ 
tion  function  P  there  exists  a  nontrivial  o-finite  invariant  measure  x.  If  a 
measure  r*  is  o-Gnite  and  invariant  with  respect  to  P,  then  x'  is  a  multiple  of 
x.  Furthermore,  the  Markov  chain  is  x-recurrent. 

Proof.  See  Orcy  (1971).  3 
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(tt)  LEMMA.  Let  X  be  recurrent  with  invariant  measure  *.  If  f  and  g 
are  measurable  functions  satisfying  f£\J]dr  <  Je  \o\dx  <  “pcOi  and 

f£gdv  >  0,  then 

g.[Iwo/(^)l  _  f£/d* 

■ 00  E*E*!-o  ?(-**)! 

for  x  £  E. 

Proof.  See  Revu:  (1975).  3 


(7)  LEMMA.  Let  X  be  recurrent  with  invariant  measure  *.  If  f  and  g 
are  measurable  functions  satisfying  f£l/]dx  <  -)-oo,  f£jgidr  <  -f-oo  and 
fEgdr  >  0,  then 


lim 

n— »oo 


EL,iM 


IfJJz 

1e  »«» 


P„  —  a.e. 


for  any  probability  measure  u. 

Proof.  See  Rev ux  (1975).  When  <?  =™  1  jr,  this  is  the  strong  law  of  large  numbers. 

a 


If  X  is  (A.  *?,  X,  l)-recurrent,  then  condition  (ii)  of  Definition  4(b)  allows 
the  decomposition  of  the  transition  function  P  by 


p .(*,*)  -  +  (1  -  p)Qi*,B)  (8) 


for  0  <  p  <  X.  The  following  critical  lemma  makes  clever  use  of  this  fact.  The 
idea  presented  in  the  proof  is  integral  to  the  future  treatment  of  the  problem, 
so  a  sketch  of  it  is  included. 

(9)  LEMMA.  If  X  is  an  (A,  p,  X,  l)-recurrent  Markov  chain  then  there 
exist,  a  random  time  N  such  that  P X(S  <  +co)  — »  1  and 

p,(*n  e  a,  n  -  n)  -  „(BO)P.(N  -  «) 
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for  all  x  E  E,  B  E  6. 


Proof.  Let  K*  ™  E  X  (0,  I),  6'  be  the  o-algebra  of  subsets  of  K*  given  by 
6  X  {0,  1  },  and  P*  be  a  transition  function  on  £*  X  S'  defined  by 

(l-p)P(*,B),  if  x£A, 

(l-p)Q(*,B),  \t  xEA; 

(10) 

pP{x,B),  itxfcA, 

Pvo(^fl^).  if  *€  A; 

where  6  —  0  or  1  and  Q  is  defined  in  (8).  Let  X'  «•  {  ( Xn ,  i„),  n  ■■  1,2,...  }  be 
a  Markov  chain  on  (£*,6*)  with  transition  function  P*.  Then  it  can  be  checked 
that 

(i)  {  A'n,  n  >0}  is  a  Markov  chain  on  (E,6)  with  transition  function  P; 

and 


!*((*,<),  <fl,  0))- 
P-ttx.iJ.fS.l))- 


(ii)  {  6n,  n  ^  0  }  is  a  sequence  of  independent  and  identically  distributed 
Bernoulli  random  variables  with  parameter  p. 

Define  N  —  inf(r»  >  l'JCt— t  E  A,6n  —  1).  It  remains  only  to  observe 
that  the  probability  the  Bernoulli  variables  are  zero  each  time  the  process 
euters  A  is  zero.  U 


It  should  be  noted  that  the  transition  function  in  (10)  is  not  the  same  as 
that  reported  in  Ath*eyn  and  Ney  (1977,  1978).  The  author  wan  notified  of 
the  changes  in  the  transition  function  by  private  communication. 

An  immediate  corollary  is  that  a  sequence  of  these  times  can  be  found. 
Unfortunately,  only  very  special  processes  arc  l)-recurrent,  but  any 

(A,<p,  X,  fc)- recurrent  pr<  .*sa  con  be  closely  linked  to  one  of  them.  There  is 

one  additional,  but  famii'ar,  requirement.  For  each  A,  define  the  set 
ka  *■  {k  >  1:  there  exists  X*  >  0,  and  a  probability 

distribution  on  A  with  the  property  that 

P k(x,B)  >  X/t^k {B)  for  all  x  E  A  and  B  Cj  A.  } 

11  the  greatest  common  divisor  of  is  1,  the  chain  X  is  said  to  be  aperiodic. 
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(11)  LEMMA.  Let  X  be  aperiodic  and  (A,p,  X,  A;)-recurrent.  Then  the 
skeleton  process  Y  “  {  Yn  —  n  —  0,  1, . . .  }  is  [A,  p,  X,  1  J-recurrent. 

Froof.  See  Athreya  and  Ney  (1978a).  3 

The  second  type  of  recurrence  is  a  global  condition. 

(12)  DEFINITION.  DOEBLLN’S  CONDITION -Doeblin  (1940).  There 
exists  an  positive  integer  n,  two  real  numbers  -j  <  1  and  tj  >  0,  and  a 
probability  measure  r  on  [E,  6)  such  that,  for  A  £  6. 

t(A)  >  7  implies  Pn(s,  A)  >  r\  for  all  iGE. 

Another  formulation  of  this  idea  is 

(13)  DEFINITION.  DOOB’S  CONDITION-Dooh  (1953).  There  is  a 
finite  valued  measure  p  on  set  A  G  6,  with  p(E)  >  0,  an  integer  k  >  1  and 
a  positive  t  such  that,  for  all  x  G  Et 

P^xM)  <1  — «  whenever  sff(A)  <  r. 

Occasionally,  the  parameters  satisfying  the  conditions  will  be  made  ex¬ 
plicit  by  stating  a  4-tuple  (r,  n,  7,  tj)  Tor  Doeblin’s  Condition  and  a  triple  (p,  k,  t) 
for  Doob's  Condition.  Note  that  p  can  be  replaced  by  a  probability  measure 
P*(A)  —  p(A)/p{E).  If  (p,  k,  t)  satisfies  Doob’s  Condition,  then  [p*,k,6)  does 
as  well,  with  6  — »  min[e/,o(E),  e].  In  this  report,  the  measure  p  will  always  be 
a  probability  measure. 

The  literature  is  somewhat  ambiguous  regarding  the  relationship  between 
the  two  global  conditions  (see  Dcob  (1953,  and  Athreya  and  Ney  (1978)).  Thr 
following  theorem  clarifies  the  situation. 
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(14)  THEOREM.  Let  X  be  a  Markov  chain  on  (E,6).  Doob's  Condition 
is  satisfied  if  and  only  if  Doeblin's  Condition  is  satisfied. 

Proof.  Suppose  Doob's  condition  is  satisfied  by  e).  For  f?€6  satisfying 
>p(B)  >  1  — c,  or  p(B*)  <  e  it  must  be  that  P^x.P*)  <  1  — c  or  P*(x,.l?)  >  e. 
Thus  Dceblin's  condition  is  satisfied  with  4-tuple  (v?,  Ar,  1  — (><)• 

Conversely,  suppose  Doeblin's  condition  is  satisfied  by  (r,  n,  *y,  rj).  If  B  G 

6,  with  r(£?)  <  1 - j,  the  condition  implies  P"(x,£J)  <1  —  tj.  This  means 

Doob's  condition  is  satisfied  by  triple  (r,  n,  6)  with  f  —  min[l  — 7,q].  3 

It  is  interesting  to  note  that  X  is  (£7,  \,  n)-recurrent  if  and  only  if 

X  satisfies  Doeblin's  condition  (See  Neveu,  1984).  For  this  reason  it  U  not 
surprising  that  the  limit  results  for  these  chnius  are  stronger  than  those  for 
Harris  recurrent  chains.  To  explore  these  results  more  terminology  is  required. 

(15)  DEFINITION.  A  set  U  will  be  called  a  consequent  set  if,  for  some 
x  G  £7,  Pn(x,B)  —  l  for  all  n,  and  in  this  case  B  will  be  called  a  consequent 
of  x.  A  set  which  is  a  coniequeit  of  every  one  of  its  points  will  be  called  an 
invariant  -t. 


Doob  has  demonstrated  that  when  his  condition  is  satisfied,  there  is  a 
decomposition  of  E  into  disjoint  invariant  sets  Ei,  E2,  . . .  and  a  transient  set 
F  —  £7  —  IJ^Ll  F,.  Corresponding  to  each  E,  is  a  probability  measure  r%  with 
the  properties  that 


»(£7,)  •—  1  and 


SZ-.p-i-.c) 

lirn  - 

»»— 00  n 


»,(C)  for  x  €  £7„C  C  £7,. 


The  £7,  are  called  crgodic  sets.  Furthermore  each  £7,  can  be  decomposed  into 
d ,  disjoint  sets  sur^  that 


P (x.D.j)  —  1  for  x  S  j  ™  1, 2, . . .  ,d,. 

The  sets  D,,;  are  called  cyclically  moving  subsets  of  the  ergodic  :  et  E,. 


11 


(IS)  DEFINITION.  If  the  Markov  chain  X  satisfies  Doob's  Condition 
and  has  a  single  rrgodic  set  with  no  cyclically  moving  subsets  (d,  »  1)  then 
condition  Do  is  satisfied. 

Geometric  convergence  to  the  invariant  measure  is  the  result  of  interest 
for  these  Markov  chains. 

(17)  LEMMA.  Let  X  be  a  Markov  chain  satisfying  Condition  D?  with 
an  invariant  measure  x.  For  some  n<,  <  -f-oo,  some  p  <  l,  n  >  n^, 

|PnC*.  A)  —  v(A)|  <  pn 

where  A£6,iGE. 

Proof.  See  Doob  (1053).  Q 

Occa  unally  it  is  convenient  to  redefine  Condition  Do  solely  in  terms  of 
the  trrnsition  function  P.  One  way  to  do  this  is  via  (he  coefficient  of  ergodicity. 

(13)  DEFINITION.  Let  (E,6)  be  a  measurable  space  and  P  be  a  transition 
probability  function  defined  on  (E,6).  The  real  number 

o(P)-l  -  sup  |P(*,i?)-P(y,i?)| 

•,V€£ 

Be* 

is  called  the  coefficient  of  ergodicity. 

DobruStn(1958)  notes  that  a  necessary  and  sufficient  condition  for  o(P)  to 
be  zero  is  the  existence,  for  any  positive*,  of  x,  y  £  X  with  measures P(x,  •)  and 
P(y,  •)  concentrated  on  sets  B\  and  Bi  with  the  property  that  P(x,  f?,  f]B{l  <  t 
and  P(y,Bif]B7)  <  «. 

For  n  Markov  chain  a  coefficient  of  ergodicity  can  be  determined  for  each 
iterate  of  the  transition  p-obability  function.  When  one  of  those  coefficients 
is  positive,  we  shall  say  that  the  Ergodic  Condition  is  satisfied. 
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(19)  THEOREM.  Let  X  be  a  Markov  chain  on  (E,6)  with  transition 
function  P.  Then  Condition  Do  is  satisfied  by  X  if  and  only  if  the  Ergodic 
Condition  is  satisfied. 

Proof.  Suppose  a(P*)  >  a  >  0.  Choose  x  E  E  and  any  set  A  E  6  with 
P*(xM)  <  a/2.  For  any  y  E  X, 

|P*(y,*)-P*(^)|<  »up  |P*(y,.4)-Pk(x,/l)|-  1-0(1*). 

•.vex 

Be* 

Therefore, 

PV/l)  <  P*(xM)  +  (1  -  o)  <  1  -  |. 

This  means  that  Doob's  Condition  is  satisfied  by  triple  (P*(x,  -),fc,a/2).  Now 
it  must  be  demonstrated  that  there  is  but  a  single  ergodic  set  with  no  cyclically 
moving  subsets.  1/  there  were  two  ergodic  srts,  they  would  be  invariant  and 
disjoint,  implying  that  af/*4)  ■»  0  for  all  k.  If  there  were  cyclically  moving 
subsets,  P*(x,  •)  would  be  concentrated  on  a  particular  one,  depending  on 
which  subset  contained  x.  This  would  imply  that  o(P*)  0  for  all  k.  Thus 

condition  Do  must  hold. 

Now  suppose  X  satisfies  Condition  Do  with  triple  (p,  fc,  £).  Lemma  17 
implies  that  for  n  >  no, 


|P"(*,*)-P*»(y,/l)|<2p'» 

for  all  choices  of  x,  y  E  E,  E  6.  The  constant  p  is  strictly  less  than  1,  :o 
off**)  >  0,  for  all  n  such  that  1pn  <  1.  J 

It  is  not  curprising  to  find  that  many  results  that  have  been  found  for 
Doeblin  recurrent  chains  have  also  been  found  for  chains  satisfying  the  Ergodic 
Condition  The  inter-ited  reader  sou'  !  see  Ioaifescu  and  Theodorescu  (1959) 
for  a  comprehensive  study  from  this  viewpoint. 


2.2.  GSNIPS  AND  RELATED  PROCESSES 


Among  the  most  interesting  efforts  to  use  stochastic  processes  as  models 
for  complex  phenomena  are  the  generalized  semi-Markov  processes  suggested 
by  Matches  (1982).  The  construction  of  these  processes  that  follows  is  based 
largely  on  the  presentations  of  Whitt  (1976)  and  Hordijk  and  Schassberger 
(1970). 

Let  S  be  a  finite  set  called  the  state  space  and  E  be  a  finite  set  called  the 
event  space.  With  each  state  »6S,  associate  a  positive  integer  n,  and  an  n,- 
tuple  of  distinct  elements  of  E, 

£»  “  («l(*)»  «?(*).  •  •  • .  *«.(*))  w«th  *,(*)  G  E. 

This  event  set  lists  the  possible  events  which  can  occur  in  state  s.  With  some 
abuse  of  notation,  we  shall  write  r,  E  E(j)  if  e,  is  a  coordinate  of  the  vector 
E(s).  Also,  with  each  associate  the  space  of  clock  readings 

c’(5)  “  {(ei.c'i, ...  ,e,E;):c*  —  0  if  e,  £  £(s),c,  >  0  otherwise  }, 

where  [Ej  is  defined  to  be  the  number  of  elements  in  E.  Each  element  of  C(s) 
has  a  coordinate  corresponding  to  each  event  in  E.  I3y  convention,  if  the  even*, 
e,  is  not  in  E(s),  the  clock  corresponding  to  e,  (c,)  will  read  0  in  each  element 
of  C(s).  If  e,  E  £(*),  then  c,  records  the  amount  of  time  remaining  until  the  e, 
occurs.  The  clock  c,  and  event  e,  are  said  to  be  active  in  state  s  if  e,  E  E(s).  Let 
G(s)™{  »  }  X  C(s)  and  W  ■—  G(s).  Let  o-algebrn  generated  by 

sets  of  the  form  (s,A),  where  «  E  £  and  A  is  a  I3orel  set  in  projected  to 

the  active  components  of  C(s).  Then  (IT,  7*)  is  a  measurable  space.  Denote  by 
(n,7)  the  product  space  ((IT)00,  (ff')<*5)  and  call  Zn  -«*  (.Vn,Cn)  the  coordinate 
mappings  of  H.  The  »th  coord,  .ate  of  the  vector  Cn  will  be  denoted  C*.,.  An 
element  of  fT  shall  be  called  a  Z-state  to  distinguish  it  from  an  element  of  S. 
Each  point  of  IT,  th  eforr,  contains  an  element  of  S  and  a  vector  of  clocks 
indicating  the  time  remaining  until  each  scheduled  event  occurs 
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To  determine  how  Z  —•  (X,C)  —  ((,Yn,  Cn),  n  0,  1,...)  behaves,  th.ee 
families  of  functions  are  required.  First  consider  a  family  of  speeds  ( kty,i  G 
5,e6A’(j))  with  the  properties 

(1)  k M  >  0  for  all  5  G  5,e  6  E(a); 

(2)  >  0  for  all  »  6  5; 

(3)  £^**..>0  for  alleGE. 

Intuitively,  A:,,  is  the  amount  the  clock  associated  with  event  e  decreases 
in  each  unit  of  time  when  the  system  is  in  state  s.  The  most  natural  value  of 
kt,  is  1.  This  means  that  the  amount  of  time  remaining  until  event  e  occurs 
decreases  by  one  unit  in  each  unit  of  real  time  that  elapses.  Occasionally, 
however,  models  require  speeds  that  are  not  unity.  In  computer  models,  for 
example,  processor  sharing  is  sometimes  viewed  as  a  server  performing  in  each 
unit  of  time  1/n  units  of  service  for  each  of  the  n  jobs  requiring  his  service.  For 
this  situation  positive  speeds  other  than  unity  are  useful  and  state-dependent 
speeds  are  required. 

When  services  are  interrupted  and  resumed  at  a  later  time,  speeds  of  zero 
are  needed.  An  example  of  a  model  with  this  feature  will  be  introduced  in 
Section  3.2.  A  speed  of  zero  is  a  technically  tricky  addition,  however,  and  we 
will  need  to  make  an  assumption  to  simplify  the  situation.  Assumption  (9) 
will  formally  state  the  requirement  we  need. 

Now  a  mapping  to  govern  how  X  changes  when  events  occur  must  be 
defined.  Let  the  family  of  mappings 

p(s,  e.s'J-.S  X  E  X  S  — •  [0, 1] 

indicate  the  probability  that  /  is  entered  from  s  when  event  e  occurs.  Assume 
that 

(4)  if  f  •,e,a/)  >  0  then  e  E  E(a); 

P)  if  p[s,  e,  s')  >  C  then  E'a)  —  {  e  }  £  E(»0; 


15 


(7)  Tor  each  pair  (s,e),  s  E  S  ,  e  G  E(«),  Yi«=s  Pi5> e*  •')  “  1  vvhen  >  °i 

(8)  for  each  pair  (i,j')65xS,  there  is  a  finite  sequence  of  events  and 

states  ei.si.ej . e„,  which  satisfies 

(9)  if  k,»  ”  0  for  a  pair  »  6  5,  e  €  JET(®),  there  is  a  path  s,  c\,  sj, . . . ,  s„ _ i , 

tn,  *n  with  the  properties  that  k^,  >  0  and  kttlf  —  0  for  i  —  1 . n  and 

e,e{«’S£-(.m„-0}. 

Property  (5)  means  that  we  will  assume  a  'noninterruptive*  scheme  (Schass- 
berger’s  terminology).  If  an  event  is  active  in  a  state  and  does  not  occur  then 
it  must  be  active  in  the  next  state. 

When  property  (8)  is  satisfied  the  process  is  uaid  to  be  irreducible.  The 
sequence  (s,  ej,  »i, . . . ,  eni  s*  will  be  called  the  path  between  s  and  s’. 

Condition  (9)  is  the  te^hniral  convenience  mentioned  above.  When  a 
speed  associated  wi  h  an  ev«  :»t  e  is  z^ro  in  a  particular  s  G  S,  this  condition 
requires  that  there  is  a  path  to  a  state  s„  where  the  speed  k^,  is  positive, 
while  the  other  clocks  with  zero  speeds  iii  state  s  a!>o  have  a  zero  speed  in 
each  state  of  the  path.  This  means  the  zero  speeds  can  be  made  positive  one 
at  a  time.  This  property  will  be  convenient  in  our  discussion  of  models  with 
zero  speeds  in  Section  3.1. 

The  scheduling  of  events  is  governed  by  a  family  of  lifetime  distributions  . 
{F(-,s,  e,  f,e');s  6  5,«£  £T(s),  <  !>  0,t?  E  £}.  The  distribution  F(s,  j,e.  t,  e') 
represents  the  probability  that  the  clock  associated  with  event  e  will  be  set 
at  a  value  less  than  or  equal  to  x  when  there  is  a  transition  into  state  s  after 
the  occurence  of  event  e7  with  the  clock  associated  with  event  c  rending  t  at 
the  instant  of  transition.  Kor  each  s  £  S,  t*  £  E,  c  G  £(*),  thesr  distributions 
must  possess  the  following  properties: 


(10)  fcr  t  —  0,  F(0,s,e,t,e')  —  0; 

(11)  Tor  t  >  0,  F(x,s,e,t,ef)  —  l[i,oolW; 

(12)  for  t  ■■  0,  have  densities  with  respect  to  Lebesgue  measure  which 
are  bounded  by  K  and  have  support  on  (0,c,,y),  a,,,lC<  <  +oo.  Also  suppose 
that  the  distributions  have  finite  first  moments.  Throughout  this  study, 

will  be  a  random  variable  with  distribution  F(-,  s,  e,  0,  tf). 

Assumption  (11)  implies  that  if  a  clock  is  active  at  the  time  of  the  oc¬ 
currence  of  an  event  it  is  not  reset  or  become  inactive  when  that  event  occurs, 
but  reads  the  same  immediately  after  the  event  as  it  did  immediately  before 
the  event. 

For  a  pair  (s,  c)  £  fT  let 

t*  ™  t*(s,  c)  =»  min  j— — 

where  the  minimization  i3  taken  over  those  coordinates  with  c,kliti  >  0. 
Intuitively,  t*  is  the  minimum  time  until  cne  of  the  active  clocks  of  s  hits 
zero.  Let  c*  *=>  e*(#,  c)  be  the  event  associated  with  the  clock  reading  t*.  The 
variables  f*  and  e*  will  denote  the  minimum  clock  reading  and  its  associated 
event  for  the  Ith  coordinate  of  n. 

A  probability  transition  function  can  now  be  defined  for  the  process  Z. 
For  a  set 

/i-i.'jxfxitL.io.f,)) 

with  tj  ■■  0  if  c,  £(*),  tj  >  0  otherwise;  define  P:lV  X  — *  [0,  1]  by 

1£{*0I 

P((s,c),A)-p(sIe,(s,c),sO  II  (13) 

j-i 

where 

fo, 

(cz  kl  ttt  ,  if  tj  £  E{s). 
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Let  {  P((s,  c),  -):s  GS.cG  C’(5)  }  be  the  family  of  distributions  that  govern 
transitions  in  the  Z  chain.  If  F(x,  s,  e,  t,  J)  is  a  measurable  function  of  t  for 
each  x  e  [0,+oo),s  e  s,e  e  E[s)trf  e  E,  then  P  is  a  Markov  kernel  (see 
Definition  2.1.1  or  Revuz,  1975).  Finally,,  let  u  be  a  measure  on  (W,  <Jr)  and 
call  it  the  starting  measure  if  P(Zo  E  >4)  u(A). 

(14)  DEFINITION.  A  process  Z  —  {Zn,n>  0  }  constructed  in  this  way 
is  called  a  Generalized  Semi-Markov  Ordered  Pair  (GSMOP). 

A  continuous  time  process  related  to  the  GSMOP  is  the  process  of  the 
most  interest;  it  will  be  constructed  from  the  GSMOP.  First  let 


n — 1 


-  e 


nun 


m,  » 


m—0 


fc.V m,t 


and 

j\'(<)  max(  n  >  0 :Q(n)  <  t  }. 

The  processes  represent  the  time  of  the  n'"  transition  and  the  number  of 
transitions  by  time  t,  respectively. 

(15)  DEFINITION.  The  process  25  —  (  Z,,  t  >  0  }  —  {  (%t,  C«),  t  >  0  } 
where 

2>i  —  (5e,  Cj)  (X,v(o»  C.V(i), l  —  kXwfO.'i^  (*)))•  •  •  • » 

Qvco.lfi!  —  —  (*)))• 

is  called  a  Generalized  Semi-Markov  Order  (GSMO).  lu  first  component  is 
the  generalized  Semi-Markov  Process  (GSMP). 

The  only  standard  terminology  here  is  the  GSMP  —  each  author  lifts 
his  own  terminology  for  the  other  proccNses.  It  is  important  to  note  that  the 
GSMOP  i ;  n  Markov  chain  on  a  general  state  space  and  the  GSMO  ir.  a  Markov 
process  on  the  same  state  space. 

An  example  will  help  clarify  the  notation. 
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Figure  1.  Two  Station  Single  Server  Cyclic  Queue 

(18)  \  SAMPLE.  Two  station  single  server  cyclic  queue  with  k  customers. 

This  system  is  an  arrangement  of  two  simple  queues  with  the  departure 
process  or  each  queue  the  arrrival  process  of  the  other.  Figure  1  shows  a 
schematic  diagram  of  this  system.  In  this  figure  (as  in  all  others  in  this  report) 
a  circle  represents  a  server  while  a  rectangle  represents  a  waiting  room. 

The  state  space  ?  Tor  the  queue  length  process  of  this  system  could  be  the 
collection  of  ordered  pairs  ((0,  k),{l,k —  1), . . . ,  ( k ,  0))  where  each  pair  (i,  k  —  i) 
indicates  that  i  customers  are  waiting  for  or  are  being  served  by  server  A  while 
k-i  are  doing  the  same  for  B.  The  only  events  that  can  occur  are  a  service 
completion  by  A  or  one  by  B,  implying  that  the  clock  vector  c  is  a  pair.  The 

first  component  is  active  in  states  ((1  ,k —  1) . (Ac,  0))  while  the  second  is 

active  in  ((0,  fc), ....  (Ac  —  1,  1)).  Supposr  the  initial  distribution  of  the  process 
is  u{A)  =—  P{  ((0,  k),  (0,  cj),  A  }.  Note  that  the  choice  of  c  is  immaterial  so  long 
as  it  is  positive. 

F  igure  2  illustrates  a  sample  path  of  GSMO  and  its  related  processes. 
The  4-tuple  (number  of  customers  at  station  A,  number  of  customers  at  B. 
time  remaining  in  service  at  station  A,  time  remaining  in  service  at  station 
B)  for  (  >  0  is  a  GSMO.  The  GSMOP  is  the  collection  of  readings  of  the 
GSMO  a'  the  times  events  occur  (dotted  lines).  The  first  component  of  the 
GSMO  h  a  GSMP.  (In  this  case,  it  is  the  order  pair  describing  the  number  of 
jobs  waiting  at  each  server.)  Notice  that  the  slope  of  each  clock  is  -1.  This 


10 


indicates  that  the  speed  of  that  clock  is  1.  The  speed  of  service  is  the  absolute 
value  of  the  slope  of  the  corresponding  dock. 

If  station  A  has  i  servers  and  station  D  has  j,  the  system  will  be  referred 
to  as  an  (i.j)-server  cyclic  queue.  An  element  in  the  state  space  S  would  be 
an  ordered  triple  consisting  of  an  i*luple  indicating  which  servers  are  active 
at  station  A,  a  j-tuple  measuring  the  same  thing  at  station  D,  and  an  integer 
between  0  and  k  representing  the  number  of  jobs  waiting  and  in  service  at 
station  A.  There  would  be  i-f-j  clocks  in  the  event  space  —  one  for  each  server. 
If,  after  completion  of  service  at  station  A,  a  job  may  reenter  the  waiting 
room  of  A  with  positiv -r  probability,  the  system  will  be  said  to  have  feedback. 

Throughout  this  report,  the  limiting  behavior  of  CSVOs  and  GSMOPs  is 
a  question  of  great  interest  and  a  note  about  the  existenceof  invariant  measures 
should  be  made  here.  Results  in  Sections  3.1  and  4.2  allow  the  classification 
of  GSMOPs  by  recurrence  properties  and  the  application  or  results  in  Section 
2.1.  Invariant  measures  exist  for  all  GSMOPs  which  satisfy  the  assumptions 
(1)-(12)  and  the  additional  stipulation  that  a,  ty  is  either  finite  or  infinite 
for  all  5  £  5,  e  E  E,  c*  £  £T(j).  Whitt  (1976)  demonstrates  the  existence 
of  such  a  measure  under  slightly  different  construction.  The  proof  can  be 
easily  modified  to  the  situation  specified  here.  We  shall  indicate  the  limiting 
distribution  associated  with  a  process  by  a  prime.  For  example,  the  limiting 
random  variable  of  Z  is  Z\  and  Z  is  Z/. 

Finally,  occasionally  more  than  one  GSMOP  or  GSMO  will  be  under 
discussion.  When  there  is  danger  of  confusion,  a  subscript  will  br  appended 
to  the  notation  to  indicate  which  process  is  concerned,  as  in  Q'vv  or  Pz. 
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2.3  SIMULATING  REGENERATIVE  PROCESSES 


In  THIS  SECTION  the  techniques  Tor  simulating  regenerative  processes  will 
be  reviewed.  A  comprehensive  study  of  the  properties  of  regenerative  processes 
can  be  found  in  Qinlar  (1975)  while  Crane  and  Lemoine  (1978)  exhaustively 
treat  the  use  of  these  proc rases  in  a  simulation  context.  Before  studying  the 
method  a  few  definitions  are  needed.  Let  P)  be  a  probability  space 

and  X  {A»,  t  >  0}  a  sequence  of  random  variables  defined  on  Q.  Also 
suppose  ■—  a[Xtts  <  d)  is  the  augmented  sigma-field  generated  by  the 
family  {  X„  t  <  d  }. 

(1)  DEFINITION.  A  random  variable  T  is  called  optional  (or  a  stopping 
time)  relative  to  X  “  (X»,  t  >  0)  if  and  only  if  it  takes  strictly  positive  real 
values  or  -}-cc  and  satisfies 

{u»:7(u>)  —  t  }  6  5|, 

for  all  strictly  positive  n  and  co.  The  pre-T  <7-field  Ft  is  the  collection  of  all 
sets  in  fl  of  the  form 

UT  <()(>>.( 

where  /\|  £  Ff,  for  all  strictly  positive  t  and  -j-oo.  The  post-T  field  is  the 
Borel  field  generated  by  the  post-T  process  {  Xx  >-t>  d  >  0  }  where  .Yt-,-i(uj)  » 
Xt(u,}-m(^)  for  all  t. 

(2)  DEFINITION.  The  sequence  5  =-  (5„,  n  =.  0,1,...)  is  called  a 

renewal  process  if  S'  —  (5,  —  1,2,...)  is  a  sequence  of  iud  -pendent 

and  identically  distributed  positive  random  variables.  The  counting  process 
for  S  is  defined  for  all  d  >  0  by 

N(d)  —  sup( n:Sn  <  t). 

An  important  result  about  the  counting  process  is  the  Elementary  Renewal 
Theorem. 
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(3)  LEMMA.  If  the  renewal  process  S  satisfies  the  property  that  S„  < 
4-00  almost  surely  for  all  n,  then 

Um  e{n(q>  _  1 

»-»oo  t  E{S„  —  5„_i } 

Proof.  See  Qinlar  (1975).  If  S  is  integer  valued,  then  the  counting  process 
N  can  also  be  integer  valued  and  a  corresponding  result  can  be  obtained  for 
lim,— »oo  E{  iVj  }/*'.  B 

(4)  DEFINITION.  The  process  {  Xf,  t  >  0  }  is  said  to  be  regenerative 
provided  there  exist  a  sequence  5g,Si,...  of  stopping  times  such  that 

(a)  S  =-  {  Sn,  n  >  0  }  is  a  renewal  process;  and 

(b)  for  any  n,  m  G  N,  f  j ,  <2 . tn  G  f?-*",  and  any  bounded  function  f 

defined  on  fln‘, 

E{  f{XSm.+ix . Xs„~tJ  |  X,.  t  <  Sm }  -  E{  f(Xtv  Xt . XJ  }. 

Intuitively,  the  second  condition  requires  that  the  process  after  Sm  be 
independent  of  the  past  and  the  probability  law  governing  the  post process 
be  the  same  as  thate  governing  the  original  process. 

EXAMPLES 

(5)  Let  {  Xn,  n  >  0  }  bea  recurrent  Markov  chain  on  state  space  (1,  2, . . . ,  k), 
with  initial  state  i.  If  Zt  Xn  for  all  t  G  [ft,  n -f*  1)»  then  Z  is  a  regenerative 
process  with  the  nth  regeneration  time  being  the  nlh  entrance  to  j. 

(6)  Let  (Zt,  t  >  0  }  be  the  queue  length  process  of  a  GI/G/1  queue,  with 
traffic  intensity  p  <  1.  The  regenerative  t»mes  are  the  times  of  these  arrivals 
which  find  an  empty  system. 
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The  idea  behind  the  regenerative  method  is  the  division  of  a  sample  path 
of  the  process  into  pieces  which  are  independent  aud  identically  distributed, 
and  the  formation  of  a  central  limit  theorem  based  on  those  i.i.d.  pieces.  Let 
{  Xt,  f  >  0  }  be  a  regenerative  process  on  R  with  a  sequence  of  regeneration 
times  {Pn,  n  >  0  }.  Let  F  be  the  common  distribution  of  an  =■»  (3„  —  /3n— l. 
and  suppose  ^{o,,}  <  -f-oo.  If  F  is  not  arithmetic  (see  Ciai&r  (1975)),  it 
is  known  that  there  existd  a  random  variable  X  such  that  Xi  X.  Here 
«■»  denotes  weak  convergence  or  convergence  in  distribution  (P(Xj  <  x)  — ► 
P(X  <  x)  as  t  — *  -|-oo  for  all  x  for  which  P(X  <  x)  is  continuous.)  Let  f  be 
a  measurable  function  from  E  to  R  and  defiue  a  sequence  {  Yk,  k  —  1,  2, . . .  } 
where 

Yk-  f  f(X.)d s 

J  <3*— l 

Three  results  are  important  in  thr  simulation  of  regenerative  processes. 

(S)  LEMMA.  The  sequence  {{Yk,ak).k  —  1,2, ...  }  is  independent  and 
identically  distributed. 

Proof.  Obvious  from  the  properties  of  regenerative  processes.  3 


(9)  LEMMA.  If  E{/(.\')  }  <  +oo,  then  r  as  E{/(X)  }  —  E{  Yx  }/E{qi  ). 


Proof.  See  Craue  and  Iglehart  (1975).  [J 

An  application  of  the  standard  central  limit  theorem  yields 


(10)  LEMMA.  Let  Zk  —  Yk  —  E{f(X)  }a*.  with  o 2  —  Var  Zk  <  +oo. 


Then 


,V(0,  i). 

0\fn 


Proof.  See  Chung  (1974).  3 
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Typically  the  variance  constant  a  is  unknown,  and  must  be  estimated. 
The  classical  estimate  is  »2  ■-  —  2rs i  2  -+■  r2^  3  where  f  — ■  Y/S  is  lie 

sample  estimate  of  E{/JY  }  and  #it|,  aj,;  and  *1,3  are  the  sample  variances  or 
{  Yrt,  n  >  l  },  {  a„,  n  >  l  }  and  their  sample  covariance.  It  can  be  shown  that 
f  — »  r  and  s2  — »  a2  with  probability  1  as  the  sample  size  increase  to  -f-oo.  A 
continuous  mapping  argument  is  all  that  is  required  in  order  to  substitute  s 
for  a  in  (10). 
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CHAPTER  m 


GSMOS  WITH  SINGLE  SETS 


3.1.  IDENTIFYING  REGENERATIVE  GSMOS 

The  purpose  ok  this  section  is  to  find  conditions  for  a  GSMO  to  be 
regenerative  ia  the  special  case  where  aMy  ■-  +c©  Tor  all  s,  e  and  t? .  We 
seek  a  sequence  of  stopping  times  where  the  future  of  the  process  at  each  time 
is  governed  by  the  same  probability  distributions.  It  would  be  convenient  if 
each  stopping  time  in  the  sequence  is  also  the  time  an  event  occurs.  This 
would  mean  that  examining  the  transition  function  of  the  associated  GSMOP 
should  tell  us  some  information  about  the  stopping  times.  In  a  GSMOP,  a 
particular  Z-state  may  not  be  visited  infinitely  often,  since  the  clocks  have 
positive  densities  on  [0,  oo).  Therefore,  we  must  find  a  set  A  O'  where 
P((r,c),R)  -  P((y,  d),B)  for  all  (x,c),  (y,d)  G  A,  D  G  SF».  (Recall  that  is 
the  <7*algebra  we  constructed  on  Cl'.) 

Note  that  if  there  is  a  clock  active  at  the  time  of  a  transition,  the  transition 
function  P  defined  in  2.2.13  guarantees  that  it  is  not  reset  at  the  time  of  any 
event.  This  means  that  the  reading  of  that  clock  determines  a  subset  of  Ilr  to 
which  the  next  Z*state  is  confined.  (Namely,  that  subset  which  has  precisely 
the  same  readings  for  the  active  clocks.)  Different  clock  readings  therefore 
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determine  different  sublets  of  O'  to  which  the  next  Z-state  is  confined.  The 
probability  that  any  clock  haa  the  same  reading  at  an  infinite  number  of 
transitions  is  0  since  the  clock's  density  is  positive  on  [0,  oo).  This  means  that 
the  event  that  governs  a  transition  from  a  Z-state  in  A  must  be  the  only  event 
active  in  that  Z-state.  This  condition  is  formalized  in  the  following  definition. 

(1)  DEFINITION.  A  GSMOP,  GSMO  or  GSMP  will  be  called  single  if 
there  exists  a  nonempty  set  S  C  S  with  l£T(*)|  ™  1  for  all  a  G  Sj.  A  state 
•  G  Si  will  be  called  a  single  state  and  the  set  G(s)  a  single  set. 

EXAMPLES. 

(2)  (3,1)  server  cyclic  queues  with  k  jobs  and  feedback.  When  all  the  jobs 
in  the  system  are  waiting  for  the  single  server  (state  ((0,0,0), 1,0)  in  the  st.ile 
space  described  in  Example  2.2.16)  the  only  active  clock  is  the  one  which 
measures  the  remaining  time  in  service  for  that  server. 

(3)  (2,2)  server  cyclic  queues  with  k  customers.  As  long  as  the  number  of 
jobs  in  the  system  rxceeds  2,  there  will  be  at  least  2  servers  active  at  all  times 
and  a  single  set  does  not  exist.  A  closed  network  must  allow  all  customers  to 
wait  for  a  single  server  in  order  for  a  single  set  to  exist. 

(4)  A  single  Sr-ver  simple  queue  with  traffic  intensity  less  than  one.  The 
state  which  represents  an  empty  system  is  the  single  slate.  The  only  clock 
active  in  this  state  is  one  which  measures  the  amount  of  time  remaining  until 
the  next  arrival.  If  an  open  network  has  a  single  input  stream  when  the  system 
is  empty,  then  a  single  set  exists. 

An  interesting  observation  is  t’  at  all  GSMOPs  with  single  sets  do  not 
satisfy  Poeblin's  condition.  (My  thanks  go  to  Phil  Heidelbcrger  for  pointing 
out  the  principle  underlying  this  example.)  Example  (2)  does  not  for  k  >  4. 
Consider,  for  a  fixed  positive  c,  the  set 
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c  >  0,el(e«  G/?'*'  } 


.4(c)  “■  {  (((I.  1)»  T.  j).  tii  cj,  e^):^  “  2, . . . ,  k] 

s  —  0,  l;  -r  —  0, 1; ci  —  cj  — 

This  set  is  the  collection  of  Z-states  which  have  the  first  and  third  servers  at 
station  A  active,  with  the  difference  in  their  remaining  services  equal  to  c. 
For  any  n  >  1,  e  >  0,  and  any  choice  of  cj,  c«  and  c,  ci  and  cj  can  be  chosen 
so  that 

(((1.  1.  0. 1  ,k—  l)ci,pi,C3,C4)  G  A(c) 

and 

P"{(((1. l*  1).  1. Ac  —  l),cl(cj,cj,c«),/4(c)}  >  1  —  e. 

In  words  this  means  that  when  2  of  the  servers  at  station  A  are  busy  with 
n  difference  of  c  between  the  times  remaining  in  their  respective  services,  w * 
can  choose  the  times  until  those  servers  complete  service  to  be  so  enormous 
that  even  after  a  large  number  of  transitions  the  difference  between  the  clocks 
is  still  c  with  a  high  probability,  since  the  probability  that  either  server  has 
completed  service  is  small.  There  are  also  choices  of  cj  and  c 3  which  satisfy 
similar  conditions  for  A(d).  As  long  as  c  /  d,  this  implies  that  the  coefficient 
of  ergodicity  is  lero.  There  is  only  one  ergodic  set  which  has  no  cyclically 
moving  subsets,  so,  by  Theorem  2.1.19,  Doeblin's  Condition  is  not  satisfied. 
It  is  easy  to  see  that  single  GSMOPs  satisfy  condition  (ii)  of  'definition  2.1.4(b) 
for  (C7(s),  O',  P((*,c),  •),  1,  l)-recurrence  if  a  G  A,  (s,c)  G  G(«).  Is  condition  (i) 
of  the  definition  satisfied?  The  following  theorem  is  helpful  in  answering  this 
question. 

(5)  THEOREM.  Let  Z  be  a  GSMOP  with  single  state  y.  Suppose  for 
some  sj  G  S,  the  path  to  }  is  (•),•(,«?,  . .  ,  cn.  y).  Of  the  events  in  this 
path,  let  the  set  {  e,,,  eH, . . . ,  ei,  },  1  — •  «•  <  *3  <  . . .  <  >*,  be  elements  of 
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E(aj).  Dehne,  for  e^C(5jj,  and  /  •—  l,  2, . . . ,  |£(*i)|, 


t'  —  — 

1  k  * 
*n«» 


*!- 


i—i 


•I 


(8) 


Let  r(j))  be  the  subset  of  C(si)  where 


cn 

***,  k 


for  j  mm  2, . . .  t  k, 


Ehm-  1  C%j  Ylrn—  1 

- 2 -  < - ! - 


N— iN 


(7) 


for  j  >  1,1  *■  1,2,3,...,  |£'(«i)|,  is  satisfied.  Then  Y[s\)  is  not  empty.  Also,  if 
c  E  ^(si),then  P*((si,  c),  G(y))  >  0. 

Note:  (a)  In  this  theorem,  k  is  either  [EUt)!  or  [£(*i)|  —  L 

(b)  In  equation  (3),  i  f#,  *s  the  time  °f  the  Jlh  event  if  the  events  that 
are  active  in  »i  are  allowed  to  occur  con^-  u lively  in  the  "der  prescribed  by 
ij,  >2,  ...,»>.  These  times  are  necessary  for  condition  (7)  which  requires  that 
the  clocks  active  in  sj  be  in  the  right  order  for  the  path  from  sj  to  y  to  be 
followed. 

(c)  The  idea  of  this  proof  is  that  if  the  active  clocks  of  state  sj  are  in  the 
proper  order  for  the  path  to  y  to  be  followed,  the  other  events  that  must  be 
scheduled  along  tae  path  can  be  scheduled  with  positive  probability  on  a  set 
that  does  ;ot  disturb  the  order  cf  events  prescribed  in  (7).  Furthermore  the 
events  not  active  in  <*i  but  activated  in  the  path  from  *i  to  y  can  be  scheduled 
in  such  a  way  that  the  path  is  follow *d  with  postive  probability. 

Proof.  The  assertion  is  trivially  true  for  n— 1.  Now  suppose  the  assertion  is 
true  for  n  =**  0,  1, ...,  m —  1.  Now  let  r.  *  m.  To  see  the  set  f(»j)  is  nonempty, 
choose  rj  G  ^(sj)  and  set  cj,  «■  0  if  e4  <2  E(*j).  To  the  remaining  positive 
elements  cf  C2  add  for  some  r  >  0.  and  setcu,  —  C2,,+ eArt|«, 
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Let  (resetting  it  if  necessary)  the  entry  for  Cjfli  equal  This  new  clock 

vector  must  be  an  element  of 

Note  that 

P~(KcO.G(v))  >  f  l((-j,e1),G(y))P((.l(el),<f(a2,c5)), 

>  f  p(*i.ei.*j)P’n_1((»5,^).C(y))H((»,,c1),<i(s},oi)). 

If  E[*l)  C  £(*|)  —  {«»,}»  then  P((»i ,  ci),  t'fsj))  “  1  and  the  assertion 
is  true  by  the  inductive  hypothesis.  If  E(sj)  g  E(t i)  —  {e,,},  the  clocks 
activated  in  state  sj  must  also  satisfy  not  disturb  the  sense  of  the  inequalities 
in  (7)  with  positive  probability.  Since  F(*»)  is  nonempty  and  the  relations  in 
(7)  are  strict  inequalities,  t'(sj)  must  contain  a  rectangle  in  the  coordinates 
of  i7(s 5)  —  (£(*i)  —  {e,|  }  }  which  does  not  disturb  the  order  of  the  clocks 
prescribed  in  (7).  The  clocks  associated  with  the  event  must  assign  positive 
mass  to  the  appropriate  side  of  the  rectangle,  since  they  have  densities  that 
are  positive  on  the  entire  real  line,  so  (7)  and  the  inductive  hypothesis  imply 
the  desired  conclusion.  Q 

This  allows  us  to  show  that  GS.MOPs  with  single  sets  are  Harris  recurrent 
chains. 

(8)  THEOREM,  Let  Z  be  a  GSMOP  with  single  state  y.  Then  Z  is 
(G(y),D!,P((y,c),-),  1, 1)- recur  rent  for  any  cG  C(y). 

Proof.  For  any  c  G  C(y)  Condition  (ii)  of  Definition  2.1.4(b)  is  trivially  satisfied 
(G(y).n',  P((y.c),  ),l,l)  since  P((y,c),^)  —  P((y,d),  4)  Tor  all  c,d  G  C(y), 
A  G  5'.  It  remains  to  be  shown  that  P,(Zn  G  G(y)  for  i-ome  n)  —  1  for  all 
a  G  H*.  Since  S  has  only  a  finite  number  of  elements  (X*  s  i.o.)  for  some 
•  G  S.  Suppose  that  for  each  c  G  C(»)  there  is  a  finite  sequence  of  measurablr 
sets  •  •  •  (Pu(«,t)  "hich  satisfies 

P(..c)Ut  6  B\,Zi  G  Bj, . .  G  2«4«.«)+ ,  G  G(y)  }  >  0.  (9) 
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Now  define 


7(1)  —  min(n:X,»  —  *), 

7(n)  —  min(n  >  7(n  —  1)  +  u(s,  Cr(, _!)):*„  —  «). 

Il  is  clear  that 


P(.,e)(Zr(.>+.iK«^tt,.llJ  G  (G(y))‘  for  all  i)  —  0. 

This  would  complete  the  proof  of  the  theorem  provided  such  a  sequence  of 
B  sets  can  be  constructed  for  any  c  £  C(«).  To  find  the  sequence  of  sets 
for  a  given  c  £  C(j),  first  suppose  all  the  speeds  of  the  process  are  positive. 
The  condition  is  satisfied  by  choosing  a  path  from  s  to  v  in  two  pieces.  First 
select  a  path  that  could  be  followed  if  all  the  events  scheduled  in  s  occurrrd 

consecutively.  Say  the  path  this  generates  is  s.ei.si . «i£(*)|,  **•  This  means 

that  the  clocks  for  events  activated  in  ej, . . .  ,ei£(,)|  must  be  set  at  a  large 
enough  value  so  that  newly  activated  clocks  occur  after  all  the  events  active 
in  s.  This  occurs  with  positive  probability  since  all  the  clocks  have  positive 
densities  on  the  entire  positive  half  line.  To  find  the  remainder  of  the  path 
from  s  to  y,  choose  the  path  between  s'  and  y  that  exists  by  the  irreducibility 
assumption.  The  clock  vector  c  satisfies  (7)  for  this  path,  so  Theorem  5  then 
implies  the  desired  conclusion. 

Speeds  of  zero  are  slightly  more  troublesome.  The  above  path  utilizing 
the  events  of  s  may  not  be  possible  if  the  speed  of  an  event  in  E(s)  is  zero 
for  every  state  in  every  path  of  |E(s)|  steps  from  s.  This  means  that  it  may 
not  be  possible  for  us  to  construct  the  first  piece  of  the  path  ns  we  did  above. 
There  must,  however,  be  a  path  from  s  to  s*  in  S.  say  *,«i,  *i, . . . ,  e,,  s*  where 
(a'  e,  £  E(a)  for  i  —  1, . .  .,j\ 

(b)  k,> ,  —  0  for  all  e  £  A  E(s)  —  {  ej, . . . ,  r;  }. 

This  path  may  be  obtained  by  examining  all  the  paths  of  length  [Efs)! 
from  s  and  choosing  the  path  which  has  the  most  elements  in  E(s)  occurring 
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consecutively  and  which  satisfies  the  appropriate  subset  of  inequalities  given 
in  (7).  To  satisfy  the  remaining  inequalities  in  (7)  the  process  should  visit 
states  where  the  events  in  (£)  have  positive  clocks.  In  particular,  it  should 
visit  those  states  guaranteed  in  (2.2.9)  where  the  speeds  of  events  in  A  are 
made  positive  one  at  a  time.  The  irreducibility  assumption  guarantees  that 
one  state  for  each  event  in  A  can  be  visited.  The  inequalities  in  (7)  are  fulfilled 
since  only  one  event  at  a  time  is  assigned  a  positive  speed.  When  all  these 
sets  have  been  visited,  the  irreducibility  assumption  implies  that  G(y)  can  be 
reached.  By  construction  Theorem  5  then  implies  the  desired  result.  Q 

It  is  not  clear  that  assumption  2.2.9  is  necessary  to  insure  the  validity 
of  this  result.  If  the  assumption  is  not  made,  more  than  one  clock  can  switch 
from  a  zero  speed  to  a  positive  one  after  an  event  occurs  and  it  becomes 
difficult  to  determine  which  of  these  newly  activated  clocks  expires  first. 

The  possibility  of  a  clock  speed  of  zero  is  responsible  for  the  complexity 
of  this  proof.  Were  it  not  for  positive  densities  on  the  entire  real  line,  the 
result  might  not  be  true.  When  all  speeds  are  unity  a  cleaner  proof  is  possible 
(see  Sectioo  4.2). 

This  result  is  one  way  to  establish  that  the  chain  will  visit  the  single  set 
infinitely  often.  The  reasoning  in  the  theorem  can  also  be  used  to  determine 
that  the  expected  number  of  steps  between  successive  visits  to  the  single  set  is 
finite.  Combined  with  the  fact  that  the  process  leaves  the  single  set  the  same 
way  each  time,  the  key  ingredients  for  regenerative  processes  are  present. 
Visits  to  the  single  set  break  the  process  into  cycles  which  are  independent  and 
identically  distributed  random  variables.  The  formalities  of  demonstrating 
this  fact  begins  with  the  definition  of  the  shift  operator. 

(10)  DEFINITION.  Let  (E,6)  be  a  measurable  space  and  let  E°°  “Ex 
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EX....  The  shift  0  is  a  mapping  of  E°°  such  that 

8:ut  —  {w„€E,  n€A/}i-»flu  —  {  idn-*-l,  «6N}. 

Define  its  iterates  by  composition:  0°  is  the  identity  and  0h  —  0o0*— 1  for  k  ^ 
1.  Each  of  these  shift  operators  induces  an  inverse  set  mapping.  These  are 
defined  by 

6~n(A)  —  {  w  £  E  |  0n(-»)  €  A  }. 

A  random  or  o-shift  can  be  defined  by 

0°(w)  ”  0'*(ii»)  on  (u>:a(u/)  —  n  }. 

The  inverse  o-shift  is,  of  course,  a“ l(A)  -■  {ui:a(u>)  £  A  }. 

Suppose  that  Z  is  a  (G(s),fy,  P((s,c),  -),l,l)-recurrent  Markov  chain.  Then 
the  successive  entrance  times  to  the  single  set  are  optional  relative  to  {  Zn,  n  > 
0  }  and  are  defined  by: 

01  —  mia(r»  >  0  |  Z„  £  C(s)),  <*i  —  0j; 

0,  —  min(n  >  0,_t  |  Zn  £  G[a)),  a,  —  0,  —  0,_i. 

Let  the  pr e^0,  and  post-0,  o-fields  be  denoted  by  and  respectively. 
For  convenience,  let  the  initial  distribution  satisfy 

*4*)  —  P((*.cM) 

for  (s.c)£G(s),  A£^. 

(11)  THEOREM.  Let  Z  be  a  GSMOP  wi'h  single  state  s  and  single  set 
G(s).  Then,  for  each  i,  the  Borel  fields  and  are  independent.  Also  the 
post-0,  process  has  the  same  distribution  as  the  original  process. 

Proof.  Suppose  A  £  0^,  and  Bj  £  1  <  J  <  fc.  k  >  1.  The  result  will  be 

established  if  the  equality 


P*(A.  Z0l+,  £  D„  1  £  j  S  *)  "  P*,(A)P*(^  eB},  1  <  j  <  k). 


is  demonstrated.  The  expression  A  ("){  A  — *  n  }  can  be  rewritten  as  A„n{  A  ■■ 
n  }  o(Z,,  i  <  n)  where  A„  G  for  each  n.  Therefore, 

P^(A,ZA+>Gfl;,l<J<fc) 

OO 

-  2  P*(A*,  A  -  n,  Zfii+j  G  1  ^  j  <  k) 

fl«0 

OO 

-  £  P*(An.  A  -  n)P*(Z,  e  a,,  1  ^  >  <  fc) 

wO 

-P5,(A)P^(Z,Gfly,l<J<fc) 

The  second  equivalence  is  justified  by  the  fact  the  Z  is  a  Markov  chain.  Q 

(12)  COROLLARY.  For  G  G  S',  P *(*“*‘(<7))  -  P*(C). 

Proof.  Define  the  set  function  P*  by 

p '(B)  -  p 

The  negative  shift  operator  maps  disjoint  sets  to  disjoint  sets,  so  P*  can 
easily  be  shown  to  be  .  probability  measure.  If  G  is  a  finite  product  set,  such 
as 

k 

G-  r){j|J",€B„,} 

J-l 

where  {  nt,  nj . n*  }  is  an  arbitrary  finite  subset  of  the  positive  integers  Bnj 

is  a  rectangle  in  O',  then  it  follows  from  the  Theorem  that  P  and  P*  agree 
on  sets  of  this  form  and  thus  on  their  finite  unions  and  complements.  The 
measure  P  is  o-finite,  so  the  same  is  true  for  every  GG?1  (sec  Chung,  1974, 
Theorem  2.2.3).  (] 

(13)  THEOREM.  The  random  vectors  {  Vk,  k  ■■  1,2,...  }  are  independ¬ 
ent  and  identically  distributed,  where  t*  *—  { >?,,  Zjt_t+i, . . . ,  Z#,  }. 

Proof.  To3how  the  vectors  are  identically  distributed,  without  lossof  generality 
\'i  and  V 2  can  be  examiued.  For  n  G  /V  and  an  n-dimensional  set  B  G  ‘Jr,,  it 


is  true  that 

{  -  €  0:03(1)  ™  f»,  (Z’ai-t-l  (•*)»  •  •  •  *  Zai-t-aj{z))  G  D  } 

-  {»oi(l->(<))  -  n,(Z,(^*(,) . Za,(0o‘(*)))  6  £7} 

-  {  ^  "  n.  (2.  M . 2o,(*0)  e  B  } 

-  3-°‘{ cox  -  n, (ZiM, .... Zn(*))  6B}. 

The  preceding  corollary  proves  that  this  set  has  the  same  probability 

as  {s:ot  =»  n,  (Zi(x) . Zo,(x))  G  B  }.  Since  ah  —  a  off-3*-1,  it  is  clear  that 

14  G  while  Vi, . . . , Vfc_j  G  50  Theorem  1 1  establishes  their  inde¬ 

pendence.  | 

(14)  COROLLARY.  The  random  variables  {  F*,  fc  "■  1,2,...  }  are  inde¬ 
pendent  and  identically  distributed  where 

ii 

n  -  E  /(*-) 

**—  »SW— iH-l 

and  f  is  measurable  on  O'. 

Proof.  If  y„(Zi, . . .  ,Z„)  =»  j  /(Z,)  then  gn  is  measurable.  For  an  open  set 
A  in  R,  let  D  —  1  *0  Theorem  13  and  the  result  is  immediate.  Q 

A  central  limit  theorem  for  a  GSMOP  with  a  single  set  is  the  next  order 
of  business.  Consider  the  random  variables  {  Fit  —  ( 'ZY/Ea)ai,Je  >  1  }  where 
EF  and  Ea  are  the  common  means  of  their  respective  sequences.  They  are 
independent  and  identically  distributed  with  mean  0  and  variance  a1.  If  o 2 
is  finite,  the  standard  central  limit  theorem  Tor  this  sequence  is 

On/t» 

The  confidence  intervals  desired  are  for  a  function  of  Z\  so  it  would  be 
desireable  if  EF/Eo  could  be  replaced  by  E/Z'.  The  next  theorem  demonstrates 
how  this  can  be  accomplished. 
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(15)  THEOREM.  Let  Z  be  a  GS.MOP  with  a  single  set  and  a  limiting 
random  variable  Z'  distributed  according  to  r.  Let  f:(Y  •-»  R  be  measurable 
and  suppose  o'  <  -f-oo.  Then 


e: 


,(n-E(/(z')}o.) 


ov/n 


N(0, 1). 


Proof.  The  definition  of  the  sequence  {ft, »  >  1}  insures  that  for  each  n>0 
there  exists  a  unique  integer  l(n)  such  that  ft(n)  <  n  <  The  cumula¬ 

tive  process  Sn  can  be  written  as 

n  i(n)  n 

-  e  /<*> e 

The  ratio  limit  theorem  (Theorem  2.1.6)  implies  that  lim„_^fooE{  Sn  }/n  — * 
E{f[Zr)  }.  The  theorem  theu  depends  bn  the  validity  of  the  statement 


lim  E{S„}/n  — •  E{  F  }/E{a}. 

First  note  that 


lim  E< 

E  /raj 

<  lim  E< 

E  /<*) 

! 

30 

•-A.-I+1  i 

"*  n-*-foo 

i 

Aim*1 

<  lim  E{  £  |/(Z,)| 


n— - oo 


<  lim  E< 


/»* 

V 


\f(Z,)\ 


(h,- i+t 


max 

0<k<n 


According  to  Corollary  4,  the  random  sums  {  Yk'.k  >  0  }  above  are  inde¬ 
pendent  and  identically  distributed.  The  assumption  that  o2  <  -f"00  implies 

that  E{  }  <  -|-oc  for  j  —  1 . n.  Therefore,  (see  Chung, 

1987) 

E{  maxi£*£n£iBA_i.M|/(Zl)|  } 


E{sn } 

lira  - -  lim  - , 

>  ■+oo  n  n— •-f-oo  n 


Y„d  P, 


{l(n)  I  n  f  m 

£'■)-£  /«-£ 


y.d  p, 


“  Z  /  Y~dP' 

{Y.)-(  Y„d  P], 

.  JKnXv 


New  {z:/(n,  s)  <  u  }  »■  {z\&,  >  A(n)  }  €:  On  the  other  hand, 

{*  |  Yv  =»  e}  £  The  independence  of  and  l{/(n)o)  is  guaranteed 

by  Theorem  11,  and  further  simplification  yields 


H  Z  w  -  m  -p(‘M  <  -lEt  mi 

|j— 1  I  v—l 

n 

-  E(  y  >  £  p(i(„)  >  „> 
-E{r}E{/(n)> 


Then 


e,} 


lim  - 

n— »-t-oo  n 


«  lim  E{  y  }E{t(n)/n} 

n— »-t-oo 
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The  process  f(n)  is  an  example  of  a  discrete  time  couuting  process,  so  the 
Elementary  Renewal  Theorem  guarantees  that 


l(n) 
lim  - 

r»— *-f-ao  n 


1 

E{ay 


Thus 


..  E<5"> 

lira  - 

r»— »-*-oo  n 


e{e;-.  n} 

lim  - 

r»— »-+-oo  n 

lim  E{  y  }  E{  f(n)/n  } 

n— *4-00 

e vn 

E{a} 


E{/(Z0}  a 


This  sequence  of  theorems  is  an  adaptation  of  random  walk  and  count¬ 
able  Markov  chain  results  (see  Chung,  1987  and  1974).  The  close  relationship 
between  the  continuous  and  discrete  processes  makes  the  final  theorem  of  the 
section  straightforward. 


(18)  THEOREM.  If  the  process  {Z,,t  >  0}  is  a  GSMO  with  a  single 
set,  it  is  a  regenerative  process. 

Proof.  The  natural  renewal  process  is  the  sequence  of  moments  the  GSMO 
leaves  the  single  set.  The  random  variables 


7(*) 


A 


mm 


represent  the  length  of  the  cycles  in  continuous  time.  They  are  independent 
and  identically  distributed,  implying  that  {  7„,  n  >  1  }  is  a  renewal  process. 
The  Strong  Markov  property  (see  Cinlar,  p.  239)  applies  to  {  Z(t),  t  >  0  }  and 
guarantees  the  second  regenerative  requirement  is  satisfied  (see  Cinlar  8.1.14 

p.  239).  a 
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The  distribution  of  each  7,  is  nonarithmetic,  since  the  distribution  of  each 
is,  and  P[pk —  Pk  —  l  <  “  1.  80  the  existing  regenerative  techniques  are 

appropriate  for  GSMOs  with  single  sets. 
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3.2.  EXAMPLES 


Ln  THIS  section,  use  of  the  regenerative  technique  is  illustrated  in  the 
simulation  of  two  G&MOPs  with  single  sets.  Both  examples  are  queueing 
networks  which  have  arisen  in  the  computer  modeling  literature.  The  first  is 
the  single  server  cyclic  queue  introduced  in  Section  2.2.  The  second  is  a  data 
management  model  with  resource  contention.  The  assumptions  made  about 
each  model  are  those  which  ease  the  calculation  of  exact  results  in  order  that 
they  may  be  compared  with  the  simulation  results.  Th«*  most  important  of 
these  is  the  assumption  of  servers  whose  service  times  have  gamma  distribu* 
tions.  The  remainder  of  the  section  is  devoted  to  a  more  complete  presentation 
of  the  models  and  a  report  of  the  simulation  results. 

(1)  Example.  Single  Server  Cyclic  Queue 

The  single  server  cyclic  queue  introduced  in  Example  2.2.19  has  been 
used  as  part  of  a  control  variables  method  for  the  analysis  of  a  demand 
paging  computer  system.  The  details  of  this  type  of  computer  system  and 
the  reasons  for  choosing  a  cyclic  queue  for  the  control  process  are  not  the 
principle  interest  here;  an  explanation  of  these  matters  can  be  found  in  Gaver 
and  Shedler  (1971).  The  effectiveness  of  regenerative  methods  in  simulating 
queueing  networks  is  our  main  concern. 

Recall  that  the  state  space  for  the  GSMOP  can  be  adequately  described 
by  an  ordered  pair— one  coordinate  representing  the  number  of  jobs  waiting 
or  being  served  by  server  A  and  the  other  representing  the  same  qua  tity  for 
server  13  (3ee  Figure  2.2.1).  The  t»o  events  are  service  completions  and  the 
clocks  governing  service  times  are  gamma  (2.1)  for  server  A  and  gamma  (3,1) 
for  server  B.  The  queueing  discipline  is  first  in,  first  out  at  both  stations  and 
the  initial  distribution  is  given  by  u(A)  — >  P(((0.  fc),  0.  e),  A),  for  any  positive 
c.  Each  Z-state  is  therefore  a  quadruple — (job*  at  A,  jobs  at  B.  service  time 
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remaining  at  A,  service  time  remaining  at  B).  All  speeds  are  unity. 

We  will  consider  five  functions  /Ji*  •— »  R  for  this  model.  They  are: 

/t(x)  “  l{0>(*i) 

Mx)  —  l(l)(*l) 

Mx)  "  l{2)(*l) 

/«(*)  —  l{j>(*j) 

/5(s)  —  3  •  1(  3  >(xi)  +  2  •  1{  a  }(*i)  +  1{  i 

The  first  four  functions  are  used  to  estimate  the  stationary  distribution 
of  the  number  of  jobs  in  service  or  waiting  at  station  A.  The  fifth  function  is 
used  to  estimated  the  expected  number  of  jobs  at  station  A  when  the  station 
is  in  equilibrium. 

The  simulation  results  for  this  model  are  displayed  in  3  tables.  Table  1 
contains  the  results  of  runs  using  /j  and  fi.  Table  2  u-rs  fj  and  /«,  and  Table  3 
uses  /j.  In  each  table  we  list  the  point  estimate  and  half  length  of  a  90  percent 
confidence  interval  based  on  various  numbers  of  cycles.  For  all  our  runs  the 
cycles  were  based  on  returns  to  the  state  (0,k).  Each  table  also  includes  an 
estimate  (d/3)  of  the  variance  term  in  the  central  limit  theorem. 

For  these  runs,  all  of  the  confidence  intervals  based  cn  1000  cycles  covered 
the  true  value  of  the  parameter  estimated.  Also,  all  of  the  variance  estimates 
were  within  5  percent  of  their  true  value. 
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I 

TABLE  3 

SIMULATION  OF  SINGLE  SERVER  CYCLIC  QUEUE 
3  jobs  -  T( 2, 1),  f(3,  1)  servers 
90  percent  confidence  intervals 


M*)  -  3 

• 1  <  3  >  (•** )  4-  2  • 

2  >(^l)  +  1{  1 

)(*0 

Cycles 

■■jQfn 

KETujHI 

a/S 

ICO 

2.176 

.0182 

3.175 

300 

2.111 

.0328 

3.942 

SCO 

2.1C9 

.0262 

3.926 

10C0 

2.105 

.0176 

3.548 

THEORY 

-  - 

2.099 

3.428 

(2)  Example.  Data  Base  Management  System 

The  second  example  used  to  illustrate  regenerative  techniques  in  queueing 
networks  is  a  data  management  facility  known  as  Data  Lauguage/I  (DL/I). 
This  model  was  developed  by  Lavenberg  and  Shedler  (1976)  and  is  depicted  in 
Figure  4.  A  fixed  number  of  jobs,  each  representing  a  data  base  call,  circulate 
in  the  system.  Note  there  can  be  more  than  one  queue  for  a  type  of  service 
and  at  the  completion  of  service  more  than  one  route  may  be  possible.  The 
choice  of  route  is  determined  by  Bernoulli  random  variables  and 

The  diagram  differs  from  one  of  a  conventional  queueing  network  in  that 
the  circles  represent  services  rather  that  servers.  The  a  services  (oo, oj,  07, 03) 
are  rendered  by  a  single  server  representing  the  central  processing  unit,  while 
the  9  service  is  performed  by  a  single  server  representing  an  I/O  device.  It 
is  assumed  that  no  two  a  services  can  be  performed  concurrently  but  that  a 
9  service  may  be  performed  at  the  same  time  as  any  a  service.  The  services 
00.01,07  and  9  are  noninterruptable,  but  the  03  service  is  suspended  at  the 
completion  of  a  9  service;  this  interruption  is  of  the  preemptive  resume  type. 

A  processor  scheduling  decision  must  be  made  at  the  completion  of  any 
a  service  or  any  9  service  when  either  no  a  service  is  being  rendered  or  03 
service  is  being  performed.  It  is  assumed  that  the  customer  whose  service  has 
been  complete  enters  the  next  queue  immediately.  The  next  processor  service 
is  the  service  having  the  highest  priority,  where  priority  is  determined  by  an 
ordering  of  the  queues  <fc,  <71,1, 9i.i,  7i,j,  and  qj.  The  processor  scheduling 
algorithm  employed  in  this  model  is  and  qj  from  highest 

to  lowest  priority. 

One  way  to  model  this  system  as  a  GSMOP  is  to  define  the  state  space 
as  a  9-tuple.  A  coordinate  is  needed  to  record  the  number  of  jobs  waiting  for 
service  at  each  of  the  9  and  six  o  waiting  rooms.  Also  a  coordinate  is  needed  to 
record  which  waiting  room  the  a  server  is  servicing  (numbered  from  1  to  6.  in 
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descending  priority),  and  an  indicator  is  required  to  help  the  system  remember 
if  an  interrupted  oj  service  is  awaiting  resumption  of  service.  Three  clocks 
are  required.  One  to  record  the  amount  of  time  remaining  in  the  (3  service; 
one  for  the  time  remaining  in  a  a<j,  oi  or  ctj  service  and  one  for  the  remaining 
03  service.  The  03  service  must  be  separated  from  the  other  a  services  because 
of  the  preemptive  resume  interruption  at  completion  of  /?  service.  After  the 
interruption,  a  new  service  receives  the  a  server's  attention  and  the  speed 
associated  with  the  03  service  is  aero.  It  remains  zero  until  all  the  jobs  are 
waiting  for  03  service.  The  data  reported  in  Tables  5,  6  and  7  is  for  the  system 
with  two  jobs,  gamma  (2,1)  distributions  governing  the  ao  and  oj  services  and 
exponential  (I)  distributions  governing  the  remaining  services.  The  binary 
random  variable  is  1  with  probability  .1,  while  rJjj  is  1  with  probability  .2. 
Suppose  the  chain  is  distributed  initially  as  if  a  service  completion  has  just 
occurred  when  all  the  jobs  in  the  system  were  waiting  for  the  c 3  service.  That 
i»,  U(A)  -  P(((0,  0, 0, 0, 0, 0, 2, 6, 0),  0. 0,  e),  A).  Cycl  es  for  this  example  will  be 
based  on  returns  to  the  state  where  all  jobs  wait  for  03  service. 

The  functions  f  Ji*  *-♦  H  we  will  examine  for  this  system  are: 

A(x)  —  l{o)(fli) 

£(*)-  !(*}(*) 

h(x)  “ 

/«(*)  “  1{  l,2)(*l) 

M ’•*)  —  2  *  2 >C=7)  4- 1<  i  >(-n)* 

The  first  three  functions  are  used  to  determine  the  probability  that  when 
the  system  is  in  equilibrium  the  a  server  services  a  job  iu  waiting  room  C,  2  and 
5  respectively.  The  fourth  function  will  be  used  to  determine  the  probability 
the  J  server  is  busy  and  the  fifth  to  determine  the  expected  length  of  queue 
in  waiting  room  <73. 

The  simulation  results  for  /j  and  fa  are  found  in  Table  5.  the  results  for 


and  /4  are  in  Table  6  and  the  results  for  are  in  Table  7.  The  format  of 
the  tables  is  the  same  as  for  the  Tables  for  Example  1. 

For  each  function  the  confidence  intervals  (90  percent)  based  on  500  cycles 
covered  the  parameter  estimated.  In  each  case  except  one  (/s)  the  error  in  the 
variance  estimate  was  less  than  5  percent  of  the  true  variance. 

This  example,  as  well  as  Example  1,  tends  to  support  the  use  of  regenera¬ 
tive  techniques  with  models  of  GSMOPs  which  have  a  single  set. 


Figure  1.  Data  Management  Model 


TABLE  5 


TABLE  6 


TABLE  7 


SIMULATION  OF  DATA  BASE  MANAGEMENT  SYSTEM 

2  jobs  -  r  (2,1)  service  at  a0,  oj 
90  percent  confidence  intervals 


Mz)  —  2  •  !{  2  }(-c0  +  1{  1  >(*7) 

Cycles 

Point 

Estimate 

A/5 

50 

.9092 

.0952 

3.051 

100 

.8211 

.0831 

5.831 

300 

.8051 

.0439 

5.493 

500 

.8380 

.0328 

4.783 

THEORY 

.8526 

4.055 
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CHAPTER  IV 


GSMOS  WITHOUT  SINGLE  SETS 


4.1.  INTRODUCTION 

UNFORTUNATELY,  the  requirement  that  a  GSMP  have  a  single  state 
means  that  the  results  of  Chapter  III  cannot  be  used  to  model  many  stochastic 
systems.  As  a  system  becomes  more  complex,  it  becomes  increasingly  unlikely 
that  a  state  is  associated  with  a  only  one  event.  The  purpose  of  this  chapter 
is  to  examine  the  possibilities  for  simulating  these  processes.  Since  GSMOPs 
are  Markov  chains  the  results  presented  in  Section  2.1  are  the  principal  tools 
available.  A  recurrence  condition  is  required  for  the  limit  results  reported  in 
Section  2.1,  so  Section  4.2  explores  the  class  of  GSMOPs  which  satisfy  the 
various  conditions.  All  that  is  required  for  a  GSMOP  to  be  recurrent  is  that 
the  densities  governing  the  clocks  of  the  process  must  have  support  on  the 
positive  half  line.  This  allows  the  application  of  the  regeneration  and  skeleton 
lemmas  (2.1.9  and  2.1.11,  respectively)  of  the  Athreya  and  Ney  presentation 
and  makes  possible  the  construction  of  a  regenerative  process  closely  con¬ 
nected  to  the  original  GSMOP.  The  central  limit  theorem  associated  with 
this  regenerative  process  can  be  used  to  obtain  the  estimates  required  for  the 
original  process.  This  regenerative  process  and  its  central  limit  theorem  a^e 
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developed  in  Section  4.4.  The  transition  function  for  this  regenerative  process 
cannot  be  explicitly  determined  usually,  so  it  is  not  satisfactory  for  simulation 
applications.  To  remedy  this  difficulty  a  way  of  developing  'quasi-cycles’  in 
the  original  process  is  proposed.  These  quasircycles  are  based  on  a  sequence 
of  Bernoulli  random  variables  which  are  independent  of  the  original  process 
and  at  the  same  time  closely  linked  to  the  regenerative  process.  The  new 
process  bae~d  on  quasi-cvcies  and  its  central  limit  theorem  are  also  introduced 
iu  Section  4.4. 

A  nice  feature  or  the  quasi-cycle  process  is  that  the  variance  constant 
in  its  central  limit  theorem  can  be  estimated.  This  problem  is  considered  in 
Section  4.5  and  a  sample  simulation  is  presented  in  Section  4.8. 

An  interesting  ergodic  theorem  for  recurrent  GS.MOs  is  developed  in 
Section  4.3.  It  is  a  strong  law  with  the  limiting  quantity  in  terms  of  the 
invariant  measure  for  the  associated  GSMOP. 

Throughout  this  chapter  all  speeds  will  be  assumed  to  be  unity.  Most  of 
the  results  are  probably  true  without  this  assumption,  but  the  introduction 
of  speeds,  particularly  zero  speeds,  tends  to  complicate  the  situation. 
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4.2.  RECURRENCE  CONDITIONS 


Since  most  complex  systems  do  not  have  single  states,  identify  ing  those 
GSMOPs  which  satisfy  a  recurrence  condition  becomes  an  important  problem. 
Example  3.1.2  demoustrales  that  Doeblin's  condition  is  loo  strong  for  many 
simple  systems.  The  difficulty  illustrated  in  this  example  arises  in  all  GSMOPs 
which  iatisfy  l£(«}|  >  2  for  all  a  E  S,  as  well  as  many  systems  where  [£(*)|  <  2 
for  some  s.  The  problem  is  that  some  Z-states  have  clocks  that,  with  a  high 
probability,  remain  active  for  a  large  number  of  steps.  This  must  be  prevented 
if  Doeblin’s  condition  is  to  be  satisfied  because  it  allows  sets  with  small  <p- 
measure  to  be  visited  with  very  high  probability  by  a  particular  Z-state.  One 
way  to  do  this  is  to  suppose  that  <  T  <  -f-oo  for  all  choices  of  *  e  and 

e/.  This  guarantees  that,  for  any  6  >  0,  there  is  a  finite  integer  in  such  that 
for  ali  (x,  c)  E  O', 

Pcs^W  £  Ur-oO  >1-*. 

Li  other  words,  all  the  clocks  initially  active  (those  positive  components  of  c) 
expire  in  the  first  m  steps  of  the  chain  with  a  high  probability.  This  is  enough 
to  guarantee  Doeblin's  condition. 

(1)  THEOREM.  Let  Z  be  a  GSMOP  satisfying  <  T  <  +oo  for  all 
a  S,  e  E  £"(»).  e*  E  £•  Then  Z  satisfies  Doeblin's  condition. 

Proof.  Observe  that  from  any  state  (x,c)  in  fT,  there  is  an  i  in  the  set 
(1,2, ... ,  |E(x)|  -f*  1)  where  e*  @  E(x).  That  is,  in  the  first  JE(x)]  -f-  1  steps, 
at  least  one  event  that  occurs  is  not  originally  scheduled  in  x.  This  implies 
that  the  l-ngth  of  the  first  k  ■  max,es£(x)  -f-  steps  of  a  GSMOP  can  be 
bounded  below  by  the  sum  of  the  k  random  variables.  Each  of  these  random 
variables  is  chosen  from  the  family  {  Y.,y:a  E  S,  e  E  £(.;),  <?  G  E  }■  (Recall 
that  when  event  P  occurs  and  the  new  state  is  s,  the  clock  associated  with 
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event  e  E  £■(«)  is  set  according  to  the  distribution  F(-, »,  e,  0,  e*).  Ytt)t>  is  a 
random  variable  which  ha*  this  distribution).  In  addition,  for  an y  positive  e, 
there  is  a  positive  integer  j  such  that  the  sum  of  j  independent  realizations 
of  Yl  ty  exceeds  m&x,(|<<a,>( y  with  a  probability  of  at  least  1  — t.  These  two 
facts  imply  that 

P(S.«)W;  mu  CfiH;  >  m«a,,,y)  >  1  —  «  (2) 

•.•.s’ 

for  all  (x,  c)  E  fl*.  Let  q  —  j  •  max^5|£'(r)|  j.  (Note  that  q  and  j  have  « 
as  an  argument  but  we  will  suppress  it.)  The  set  ( Qq  >  max aify)  is  a 
subset  of  ((x,  c)£[x)  2  Uj— of*)  50  *he  latter  set  has  a  probability  exceeding 
1  —  e  as  well. 

For  D  E  ,  define  D ,  ■-  ((x,c):x  ■—  »,)  for  i  —  1,2, ... ,  |S|,  and  note  that 
13  “  and  the  *  are  disjoint.  It  must  be  true  that 

P«((x.c),*)  -  P*((X,C ),B  I  E[x)  Q  Uj-oO  £  Uj-oO 

+  r>({x,c),B  j  E(x)  2  UJ^0Pf.,c)(£:(x)  2  UJ.^) 

.s' 

<  V  P*((i.  c),  B,  I  £(*)  £  UJ_0'")  +  <•  (3) 

•—1 

A  reference  measure  must  now  be  constructed  to  take  advantage  of  the 
decomposition  demonstrated  in  (3).  Let  m,  be  Lebesgue  measure  on  /?*.  With 
a  slight  abuse  of  notation,  let  us  write  for  the  Lebesgue  measure  of  the 

projection  of  the  positive  components  of  £?,’ s  clocks  on  R  Since  we  are 

assuming  that  <  T,  we  have  m,(C(s,))  <  T  CW. 

The  condition  on  each  summand  in  (1)  is  an  important  one  since  it  means 
that  all  the  clocks  active  in  s,  were  activated  at  some  step  in  the  path  from  x 
to  l,.  This  prevents  the  situation  illustrated  by  Example  3.1.2  when  Doeblin's 
condition  was  not  satisfied.  Therefore  we  must  have  that 

P*((*,c),B.  |£(i)  £  UJ-o'D  <  K  l*W,m.lR).  (4) 
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Define 


M(D)  — 


_1 

1*1  m.(CK))- 


Recall  that  C(#,)  is  the  collection  of  all  possible  clock  readings  when  the  process 
is  in  state  s,.  Note  that  M  it  a  probability  measure  on  It  is  nonnegn- 
tive  and  the  empty  set  has  measure  jero.  For  pairwise  disjoint  D%  E  O',  i  — 
1,2,...,  define  DKJ  —  ((i,  c)  E  —  tj).  Then 


15! 


A'<UC->“jsjE 

*— 1  11  1 


mA£->D'A 


Is! 


Y  1  Y  m;P«j) 

“  a  isi  a  m,(c(,;)); 
-  f]  A/P.) 


Furthermore,  it  must  be  that 


m,(B,)  <  mt(C(s,))  •  |5j  •  (5) 


From  (3),  (1)  and  (5)  we  have 
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151 


«"((*.» ).m  <  v;  p*((*.«  ),b.  i  e(i)  c  U!_o«")  + « 


•—1 

IS! 


<  £  +  e 


i—i 


<£>'«*>  m,(C(..))|S|.V/[B)  + 


I— t 


<^|B(h)i  max  m,(C7(».))  A^(^)  |5|a  -»-  e 

1;S*£I*1 

<  (/cr)m**iE(-)i  |5|l  (v/(B)  + 1. 

There  is  a  6  >  0  such  that  76  <  1  -  6/{KT)m^  £<**>;|S|2A/(fl).  Pick 
6,  then 

M(B)  <  f/(K’r)m“|£(#<)l|5|aM(fl) 


implies 

P*((*,e),B)  <76  <  \-6!(KT)^b^\S\7M[B). 

The  chain  therefore  satisfies  Doob’s  Condition.  D 


For  those  GSMOPs  whose  clocks  have  positive  density  on  [0,  oc),  there 
are  many  Doeblin  recurrent  chains  which  are  closely  related.  Define  a  new 
chain  V  in  the  following  manner.  Let  the  state  space  S  and  the  family  of 
transition  functions  {  p(a,  e,  e']  }  be  the  same  in  the  two  chains.  Change  only 
the  densities  of  the  clocks.  Choose  and  U  E  R'*’  and  define  the  new  densities 
9,.».e  for  V  by 


9.,c  .',!/(*) 


.0, 


if  0  <  x  <  U; 
if  V  <  x. 


(7) 


Then  V  satisfies  Doeblin’s  condition  and  is  ,  t  X,  n)-recurrent,  for  some 
and  n.  We  shall  call  V  the  truncated  Markov  chain  of  Z.  Using  this 
truncated  chain,  we  can  show  that  Z  is  a  Harris  recurrent  chain. 
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(8)  THEOREM.  Let  Z  be  a  GSMOP  with  a,,f>,  —  +oo  for  all  »eS, 
e  G  £(«)  and  e*  G  £•  Then  Z  is  (A,<p,  X,  n)-recurrent  for  some  choice  of  A,  <p> 

X  and  n. 

Proof.  Choose  a  U  G  and  form  the  truncated  chain  V  using  the  densities 
in  (6).  For  A  G  'TV*  *0  £  we  c*aim  that,  for  n  1,2,..., 

lx.GOV, »  -  l,2,...n)  -PS(*,A). 

Since  both  P£  and  P"  are  finite,  we  need  only  consider  sets  A  of  the  form 

b-{5'}x{x^1|0,<,)} 

where  s'  G  S  and  t}  —  0  if  e}  &  £(»),  U  >  t}  >  0  otherwise.  We  will  prove 
the  claim  by  iuductiou.  Suppose  n  — l. 

:£(»0! 

Pv((«.  e),  /*!)  —  p[s,  c*(«.  e), «')  JJ  F v(*,.  *},  «0 

j-i 

—  p(s,e*(s.c).0  II  II  f, .  0 

The  functions  in  the  first  product  are  all  indicator  functions  and  once 
(s,c)  and  B  are  given  they  are  fixed.  The  functions  in  the  second  product  are 
those  clocks  which  are  turned  on  when  the  process  reaches  s'  after  leaving  *. 
To  prove  our  claim,  we  need  only  worry  about  the  distributions  in  the  second 
product,  all  other  terms  in  the  expression  are  fixed.  We  can  rewrite  each  term 
in  tue  second  product 

Fv(ejt  •>,  <„  0, 0  -  Pv(Y.,,y  <  t,) 

-  P z[Y.,^  <  <  V) 

-  P z(Y.^,  <  tj  |  Yt.,s  <,  U) 

-  P z(Y.,'S  <  (j  |  n,.y  <  V  for  all  e  G  £(»)). 
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The  last  equality  is  true  since  the  T's  are  all  independent.  Therefore  the 
assertion  is  true  for  n«"l. 

Now  suppose  it  is  true  for  n  —»  1 . m  —  1.  We  must  show  it  is  true 

for  n  ■—  m.  We  can  write 

P7(ao,£  |  e  rtV,  I  — 


I  e  nV,  I 

|  e  nv.  * 

dsi). 


!»•••»  *^*)P(fl9»  d^i  |  !|  g  n^,  i  ■■  ii  •  •  •  i 

1, 2, ....  m  —  1)  P(jo,  dzx  |  z{  G  ft'y) 


NVe  conclude  the  proof  be  choosing  n  so  that  V  is  (Cl'v,<p,  X,  n)-recurrent 
for  some  •p  and  X.  Then,  for  z  G  0y,  A  C  Q'v, 


PZ(*,£)  >  P*(:,  B  I  G  n'„.  i  -  1. 


.n) 


-  Pfa.S) 

>  M.*). 


Therefore  Z  is  (fl'v,  p,  X,  n)- recurrent.  3 


4.3.  ERGODIC  THEORY  FOR  GSMOS 


The  development  OK  au  ergodic  theorem  for  GSMOs  associated  with 
recurrent  GSMOPs  is  the  topic  of  this  section.  The  main  results  are  Theorems 
1  and  7.  The  first  is  a  limit  theorem  for  the  counting  process  generated  by 
a  GSMO.  The  second  is  a  continuous  time  strong  law  that  relates  the  limit¬ 
ing  behavior  of  a  GSMO  to  the  invariant  measure  of  its  associated  GSMOP. 
Throughout  this  section,  it  will  be  assumed  that  the  GSMOP  Z  is  recurrent 
and  has  t  as  its  invariant  measure.  Define  a  function  /:0*  — *  R  by 

/(*. c)  “  «n'“  e.  •  9(*) 

*iX> 

where  g  is  a  measurable  function  from  S  to  R.  Let 


aud 


A*  —  E,{  min  Co,  } 

Q,t>o 


"-E  r{f[Zo)}. 


(1)  THEOREM.  Let  Z  be  a  GSMO  with  counting  process  N(t).  Then 


..  m 

Inn  — ■ — 

•— "-f-oo  ! 


1 


P*  —  o.e. 


for  any  probability  measure  i/. 

Proof.  By  the  definitions  of  N(t)  and  Q(a)  (see  Section  2.2J 


<?(;V(f))  <<<<?(, V(t)-fl) 


or 


<?(*(<))  ^  t_  QjSjl)  +  1) 

N(t)  ^  N(t)  ^  N[t) 


(2) 
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wheu  N[t)  >  l.  Since  Q(n)  «■»  minc<(>0  C,Jf  the  Strong  Law  of  Large 

Numbers  for  Markov  chains  (Theorem  2.1.7)  implies 

<?(") 

lirn  - —  E„{  min  Co,  } 

n  G#c>o 

=  p  P„  —  a.e. 

for  an y  probability  measure  u.  The  Strong  Law's  Hypotheses  are  satisfied 
since 


k  <  max  E  (  ^(s,  e,  e/)  }  max  g(s)  <  -|-oo. 

«,«y  $es 


Since  limi__r.00iV(t)  —  -j-oo  , 


lim 

t— ♦fOO 


gjMO) 

m 


p  P„  —  a.e. 


(3) 


and 

<?(.v(0  +  1)  <XM(t)  +  I)  N(t)  +  1  _  ... 

I — oo  N(1)  ,-:"L  ,v(i)  + 1  '  ,\'(()  p-  ae-  (•') 


for  any  probability  measure  u.  The  result  is  then  immediate  from  (2),  (3)  and 

(•*)•  J 


The  following  two  theorems  lay  groundwork  for  the  ergodic  theorem. 
(5)  THEOREM.  Let  Z  be  a  GSMOP  with  invariant  measure  w.  Then 


lim 


l/(Zn)| 

- —  o 

n 


P„  —  a.e. 


for  any  probability  measure  u. 
Proof.  The  Strong  Law  implies 


lim  _L- £|/(Z,)|  -  Hm  i"2  W)l 
-»-*-ao  n  -f-  1  *“■  n-»-+-oo  n 

*—0  »=-0 


«^+oo  ..  M 
—  E.{  |/(Z.)|  }  P.-a.e. 
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Then 


|/(Z„)|  ,  *»  «-l 

u2  - lim  -(Daz.)|-£i/(zji), 

•»— *-4"0o  n  r*— •-foo  n  t- i 

i-"0  i—0 

_  lim  S±i  U»  lim  ggy»!l 

•»  •  f  oo  n  bm  i  oo  n  "I*  1  w  >  i  oo  ti 

*■0  P„  —  a.e. 

for  any  probability  measure  v.  | 

(6)  THEOREM.  Let  So  — •  (1,  C)  be  a  GSMO  whose  associated  GSMOP 
is  recurrent.  Then 

y«M<l+D ,(<£,)  it 

lim  - - - —  0  P„  —  c.e. 

t—-t-oo  t 

for  any  probability  measure  u. 


Proof. 


lim 

t—  -^oo 


lim 

I— •-+-0O 


9(1.)  <“  -  <“  ~  /iv(0|S(®>)  ■"! 


<  lim 
”  l-»+oo 


|/„<S<V(,,-‘,9(S,)rfl-J?<’V(',)9(5,)^| 


4~  lim 

l-*+oo 


2/wM.r11  «*<)!<«  N(l) 


<  lim 
(-•-*-00 


_  2  Iim  „m  m, 

I— hoo  N(t)  1  —  4-  oo  f 


The  last  equality  is  true  from  Theorem  3  and  the  fact  that  limi_oo.V(l)  = 
-f-oo.  Q 
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The  next  result  in  this  section  is  a  strong  law  for  Z>.  It  is  unusual  in  that 
the  limiting  result  is  in  terms  of  a  discrete  process,  Z. 


(7)  THEOREM.  Let  Z  be  a  GSMO  whose  associated  GSMOP  is  recurrent. 


Then 


lim 


*=  -  P„  —  a.c. 
M 


l-»+oo  t 

for  any  probability  measure  t/. 
Proof.  It  is  clear  that 

/.'as.)*  _ 


N(l)  + 1  (  i 


Since  .N  [t)  — *  poo  ns  t  *  -poc,  the  Strong  Law  of  Large  Numbers  (Lemma 
2. i .7)  and  Theorems  5  and  0  imply  that 


lim 


Jo'g(S,M‘ 

t 


K 

—  P„  —  a.e. 


for  any  probability  measure  u.  Q 
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4.4.  SUPPLEMENTARY  YARIADLES  AND  GSMOS 

At  THIS  POINT  we  are  ready  to  tackle  the  problem  of  developing  a  central 
limit  theorem  for  GSMOs  which  do  not  have  a  single  set.  We  will  attempt 
to  exploit  the  recurrence  properties  of  the  associated  GSMOP  to  construct  a 
regenerative  process  that  is  very  similar  to  the  GS.MO  being  studied.  The  im¬ 
portant  results  for  this  approach  are  given  in  Lemmas  2.1.9  and  2.1.11.  Using 
the  techniques  in  the  proof  of  Lemma  2.1.3,  it  is  easy  to  find  a  regenerative 
process  that  is  closely  related  to  any  GSMO  when  its  associated  GSMOP  in 
aperiodic  and  (A,^,  \,fc)-recurrent.  Unfortunately,  this  regenerative  process 
is  not  welt  suited  for  simulation  applications  because  part  of  its  transition 
function  usually  cannot  be  determined.  To  avoid  this  difficulty,  we  propose 
a  new  approach  to  simulating  the  original  process  by  using  an  independent 
sequence  of  Bernoulli  random  variables  to  break  a  sample  path  into  'quasi¬ 
cycles.  A  centra*  limit  iheoen>  based  on  these  quasi-cycics  will  be  developed 
in  this  section  and  the  estimation  of  the  i.s  variance  constant  will  be  discussed 
in  Section  4.5. 

Suppose  the  Markov  ch?in  7  on  is  aperodic  and  (A,yw,  X,fc)* 

recurrent  with  transition  'unction  P  and  starting  mrasure  p.  From  Lemma 
2.1.9  and  Lemma  2.1.11  it  is  possible  to  decompose  Pfc  in  the  manner  demon¬ 
strated  in  Equation  2.1.8,  that  is. 

P*(T,  A)  -  p^(AOR)  -h  (1  -  p)Q*(*.B). 

Now  dehnr  a  new  process 

T  -  {  Tn  =  ( Vn.  *n,  ln,  Gn):n  -  0,  l, . . .  } 

on  =.  *=  fl’  X  {  0,  1  }  X  {  lt  2, . . . ,  k  }  X  lY  Suppose  that 

P'(T0G  (D.6XC))  -  10  >(*)!<  *)(0veMnenC) 
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and  let  the  transition  function  P*  of  T  be  defined  by 


i{l>(7)iu-i>(0l{c}(y)P(t/,B) 


for  j  2, 3, . . . ,  fc;  v,y  E  E;  $ 

=**  0  or  1. 

ye  a, 

pi(.)('M^nanc). 

ye  a-, 

(l-p)i<M(l)P(v,B), 

ye  a, 

ji-p)i(*i(w*,Bnc), 

y  €  A. 

(i) 


This  transition  function  breaks  T  into  groups  of  k  steps.  If,  at  the  begin¬ 
ning  of  a  ’cycle’  (that  is,  when  l  «—  k),  Vn  is  not  in  A,  then  each  of  the  next  k 
transitions  is  determined  by  the  transition  function  P  of  the  original  process 
Z.  If,  when  /  A:,  Vn  is  an  element  of  A,  the  k-cycle  has  two  pieces.  The  first 

k  —  1  transitions  are  determined  by  P  again,  but  the  last  transition  in  the 
cycle  is  either  or  Q*  according  to  whether  the  random  variable  fn—k  i*  1 
or  0.  If  Q*  is  chosen  the  process  must  remember  where  it  was  k  steps  in  the 
past  ( Gn ).  This  is  the  most  recent  state  of  the  skeleton  process.  Note  that 
when  l  =  k,  the  first  anJ  fourth  coordinates  of  the  T  process  are  the  same. 
This  resets  the  fourth  coordinate  which  indicates  the  last  state  V’s  skeleton 
process  has  visited. 

In  a  manner  analogous  to  2.1.9(i)  and  (ii)  it  can  be  shown  that 

(2)  {  V'„,  n  —  0,1,...}  is  a  Markov  chain  with  transition  probability 
function  P  and  initial  measure  v?;  and 

(3)  {  6n,  n  =»  0,  1, . . .  }  is  a  sequence  of  independent  and  identically  dis¬ 
tributed  Bernoulli  random  variables  with  parameter  p. 


It  is  easy  to  see  using  the  reasoning  of  Theorems  3.1.11  thru  3.1  15  that  a 
sample  path  of  T  can  be  broken  into  independent  and  identically  distributed 
pieces  and  that  functions  of  these  pieces  are  also  i.i.d. .  The  stopping  times 
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required  for  these  results  would  be  those  coordinates  when  V  is  distributed 
according  to  <p. 

If  the  Markov  chaiu  Z  —  (X,  C)  is  also  a  GSMOP  on  fl',  then  Vn  ->» 
(C/n,  Kn),  where  Un  £  5  and  Kn  £  C(C/„)  for  all  u.  A  continuous  version  of  T, 
say 


r 


((=U,  36),  6,  e,  Q)  -  ((("U,,  36,),  6t,  tu  Q,),t  >  0), 


can  be  constructed  in  the  same  way  as  the  GSMO  %  *■»  (96,  C)  is  constructed. 
Using  the  reasoning  developed  in  Theorems  3.1.11  through  3.1.16,  it  can  be 
shown  that  T  is  a  regenerative  process  with  the  regenerative  times  being 
those  epochs  when  Y  is  distributed  according  to  p.  This  means  that  Y  has  a 
limiting  random  variable,  say  Y'  —  (CU,1  361).  Furthermore,  since  Vn  has  the 
same  distribution  as  Zn  (both  are  Markov  chains  on  fl',  with  the  same  starting 
measure  and  transition  function),  it  must  be  that  Yt  «■■  ('ll,,  36,)  has  the  same 
distribution  as  2,,  and  therefore  Y'  has  the  same  distribution  as  2'. 

A  standard  central  limit  theorem  can  be  written  for  <T.  Supjjose 


0o{T)  -  0 

0}[T)  —  min(fcn  >  i3^(T):6kn  —  1,  €  A,  n  >  1) 

wn- 1 

Yk  —  min  Knjg(Un) 

K"'>0 


Ofc 


T) 

£ 


min  fC-, 
o 


(4) 


for  k  =■  1,2 .  The  central  limit  theorem  is  then,  if  E{  }^(cllr)|  }  <  4-co 

and  a2  =  Var  (Yk  —  E{  j(Tl')  }ak), 

EZ-iW- 


.V(0, 1). 


(5) 


We  can  substitute  E{y(3a')  }  for  EI^Al*)}  and  we  do  so.  The  technique  for 
estimating  a  in  this  central  limit  theorem  would  be  the  standard  regenerative 
estimate  given  in  Section  2.3.  Let  ry,  s^,  and  ty  be  the  estimates 
based  on  T  for  the  obvious  quantities. 

Theoretically,  our  construction  is  now  complete.  VVe  have  constructed 
a  regenerative  Markov  chain  for  an  arbitrary  aperiodic,  [A,<p,  X,  fc)-recurrent 
GSMOP  Z  which  can  be  used  to  estimate  functions  of  the  limiting  random 
variable  of  Z’s  associated  GSMP.  Unfortunately,  in  order  to  simulate  d*  one 
needs  to  know  p  and  Q*,  or  at  least  how  to  sample  from  them.  The  problem 
can  usually  be  constructed  so  that  p  is  known.  The  measure  Q*  presents  a 
problem,  however.  We  only  know  its  form  through  its  relationship  with  p  and 
P*.  In  practice  calculating  an  explicit  expression  for  Pfc  is  usually  hopeless 
and  this  makes  the  outlook  dim  for  the  generation  of  Qk.  For  this  reason  it 
is  difficult  to  use  T  as  a  means  cf  estimating  functions  of  96'. 

There  are  certain  features  of  the  T  process  which  suggest  other  means 
of  estimating  95*  through  simulation.  As  we  have  noted  in  (2)  and  (3),  V  is  a 
Markov  chain  with  transition  function  P  and  6  is  a  sequence  of  independent 
and  identically  distributed  Bernoulli  random  variables  with  parameter  p.  It 
is  importaut  to  note,  however,  that  V  and  6  are  not  independent  sequences. 
It  is  the  values  of  the  6  sequences  which  determine  the  choice  of  Qi,  or  p  at 
those  transitions  when  the  cycles  of  T  begin  and  end.  Suppose  that  instead 
of  using  correlated  sequences  V  and  6,  two  independent  sequences  are  used. 

Suppose  that  at  multiples  of  k  steps  during  the  simulation  of  the  original 
process  Z,  au  independent  Bernoulli  random  variable  with  parameter  p  is 
generated.  C:-.ll  the  variable  generated  at  the  nXlh  step  Suppose  that  the 
elements  of  the  \  “■  {  Xnk .  n  —  0,  1, . . .  }  sequence  are  mutually  independent 
and  are  used  to  define  'quasi-cycles’.  We  will  base  a  central  limit  theorem  for 
SB  on  these  quasi-cycles.  Toward  this  end,  define  the  new  stochastic  process 
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Q  ■■  (Q«  “  [Zm  X«*  Jn),  n  —  0, 1, . . .)  with  initial  measure 


Pq(Qo  e  (A,x>j))  -  P (2b  e  ^ji(  i  >(x)i{fc>0) 


and  probability  transition  function  given  by 

Pq((*.  X.».(*.7,0)  “  P(**^)l{t>l{  x)(7)  for  j  —  2, . . .  ,k 
Pa(M,l  ),(/*,  7,  <))  -  [ft +  (1  -p)(l-7)ll{a>(«)P(.'.>\). 

Also  define  the  random  variables 


A(Q)  -  0, 

A(a)  —  min{  nk  >  A_,(Q)  |  x«*  —  1,  ZK»-i)  6  n  >  1  }, 

“  Y1  min  Cnjff(XB), 

C*'>0 


A(0>-1 

E 


(Q) 


tnin  Cni. 
c„>o  1 


At  this  point,  it  is  technically  convenient  for  us  to  suppose  that  Z  is 
Doeblin  recurrent  instead  of  Harris  recurrent.  To  do  this  we  will  suppose  that 
the  densities  are  truncated  at  U  in  the  manner  we  discussed  in  Section  4.4. 
This  means  that  the  length  of  the  quasi*cydes  are  determined  solely  by  the 
Bernoulli  random  variables  since  A  —  O'. 

(7)  THEOREM.  Let  Z  be  an  aperiodic,  (fT, <p,  X,  fc)-recurrent  GSMOP 
whose  associated  GSMO  Z  —  (2,  C)  has  a  weak  limit  %'  —  (£',  C*).  If  g  is  a 
measurable  function  from  S  to  R  and  E{  }  <  oo,  then 

£g.ik-Ei»glM  _  N(0t  „ 

Oy/n 

where  o  is  defined  in  ^4). 
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__ — 1_ 


Proof.  The  proof  will  rely  heavily  on  the  central  limit  theorem  for  the  T 
process,  so  we  begin  by  studying  that  theorem.  Define 

Note  that  R{J3i,(T))  — »  k  and  that  R(n)  Ac  if  and  only  if  0*(T)  <  n  < 
Now  rewrite  the  central  limit  theorem  for  T  as 

iZirCLp _ (minK,,t>o  *'\  ?(-*-' ) }  m‘px«,t>o  ^w»»)  ^  y  ^ 

Oy/n 

The  random  sum  can  be  replaced  by  a  fixed  sum  by  noticing  that 


Km*  9(U~)  -  E{ }jtu«,k„4>0 
<J\/R(t\) 

iKmMUm)  ~ E{  fK1')  }  m,DK-.»>0  ^  rm' 

OyjR[n) 

■  }nun*„,>o  l<mt' 

oy/R(n) 

(8) 

and,  for  any  r  >  0, 

E^j/»(.,(min*-..>o  —  E{g(^;  }minK^t>o  !<">) 

ay/R(n) 

^  Z]^^(^niin>c,t>o^m.l7(fAn)l  +  E{|'7(960I 
_ 

^  max^s2|/(*)|  X^£,(!jr)  ntin^t>o  Km, 

*  ^  7*W 
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Since  (#* (*y+i)(T)  —fa(n)[T))  <  oo  and  Y <  -f-oo  with  probability 
1,  it  must  be  true  that 

lim  p/»ww  s  \  ,  (10) 

\  °  >/K(n)  / 


Equations  (8),  (9)  and  (10)  imply 


o(lnioK^,>0^»n.g(t-'n)  ~  )  }minK_t>0^mJ  ^  N(0  l) 

o\/R(n) 

when  used  in  conjuction  with  the  so-called  'Converging  Together'  Lemma 
(see  Chung  (1974)  p.  92  or  Billingsley  (1968)  p.  25)  and  the  fact  that  R}  — •  oo 
as  j  — ►  co. 

We  have  noted  that  V  is  a  Markov  chain  with  transition  function  P  and 
initial  measure  u.  The  GSMOP  Z  is  also  such  a  chain.  This  implies  that  g{U}) 
and  min can  be  replaced  by  g[X})  and  minc^,>oC’mt>  respectively. 


Now  define 


VV(fc)-ma*{;:0;(Q)<fc}. 


Since  X  is  (O'.  X,  lc)-recurrent  (here  is  the  point  where  the  assumption 
of  Doeblin  recurrence  makes  things  a  bit  easier),  the  sequences  6  and  x  are 
both  sequences  of  independent  and  identically  distributed  Bernoulli  random 
variables  with  parameter  p.  This  implies  that 

— *  1  with  probability  1 

and  W(k)  can  rrplace  R(k)  in  the  central  limit  theorem.  This  yields 

El-.  -E(  ?(%-)  )  mmcx.  U,  _  , , 

o>/VV'(n) 

In  the  same  way  that  the  random  sum  for  the  V  process  was  replaced 
by  a  nonrindom  sum  above,  the  nonrandom  sum  for  X  can  be  replaced  by  a 
random  sum  and  the  theorem  is  proven.  3 
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The  only  obstacle  remaining  to  our  use  of  the  independent  sequences  Z 
and  x  **  nu  explanation  of  how  to  estimate  a.  This  topic  is  important  enough 
to  be  dealt  with  separately.  We  shall  address  it  in  Section  4.5. 


4.5.  VARIANCE  ESTIMATION 


In  MANY  SIMULATION  problems,  particularly  those  concerned  with  deter¬ 
mining  confidence  intervals,  the  estimation  or  a  variance  constant  is  one  of 
the  trickiest  problems  that  must  be  resolved.  Our  situation  is  no  exception. 
The  variance  constant  in  4.4.5  is  for  the  regenerative  process  T.  If  T  can  be 
simulated,  a  could  be  estimated  in  a  straightforward  fashion.  As  we  discussed 
in  the  last  section,  however,  it  is  usually  impossible  to  determine  the  transi¬ 
tion  function  Q so  we  must  find  an  alternative  approach  to  the  problem.  Ln 
section  4.4  we  returned  to  the  original  process  and  partitioned  the  sample  path 
into  ‘quasi-cycles’  using  an  independent  sequence  of  Bernoulli  random  vari¬ 
ables.  The  question  we  address  in  this  section  is  how  to  estimate  the  vari.-.ice 
constant  a  using  this  sequence  of  random  variables.  Before  we  being  recall 
that  Z  {X,  C)  is  an  aperiodic  (Cl',  X,  fc)-recurrent  GSMOP  with  truncated 
distributions  and  transition  function  I*.  The  process  T  is  a  regeenrative  process 
based  on  a  decomposition  of  P*.  Finally  the  process  Q  is  the  original  process 
Z  augmented  by  an  independent  sequence  of  Bernoulli  random  variables  x- 
Also  recall  that  f  is  a  function  defined  on  Cl'  by 

/(x,c)  —  min  c,  •  g(z) 
c«X> 

for  some  measurable  function  g:S  —  R. 

The  most  naive  approach  to  estimating  n  using  Q  is  to  form  the  same 
statistics  based  on  its  quasi-cycles  as  one  would  ordinarily  form  for  a  regenera¬ 
tive  process.  Therefore,  let 
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*™  ~ _ j  p*  **)*» 

*fa  n  _  j  ^ 

fa  —  r/u, 

*q  *™  *?i  —  2rQ«^3  -f-  »’a*?2 

where  the  { L} }  and  {  v}  }  sequences  are  defined  in  4.4.6.  In  all  of  these 
statistics  n  is  a  suppressed  argument.  To  determine  the  asymptotic  properties 
of  these  statistics,  let  us  couple  the  process  T  with  the  sequence  x-  Let  rjj  « 
[T,  x)  “  {  —  (T'n.  X««):n  —  0,  1, . . . ,  }  be  a  process  defined  on  5  X  (0,  1) 

with  initial  measure  u(A,  f)  -■  P(7o  £  I  )("T)  and  probability  transition 
function 

P*{(v.M,y.7),(^,x)}  —  P'Uv.M,  y).^}i{T>(x) 

for  (u, 6,1,  y)  G  s,f  "  2,3, ...  ,k 

Pw{(v,i,  l,y,*T ).  X)  }  “  bx  +  (1  —  p)(i  —  x)lP{(M.  l»yM). 

where  P*  is  the  transition  function  for  T  defined  in  4.4.1. 

Notice  that  ^  is  the  process  T  supplemented  by  an  independent  sequence 
of  independent  Bernoulli  random  variables.  The*e  random  variables  are  gen¬ 
erated  at  the  same  times  and  with  the  same  parameter  as  the  6  process  in 
T.  Recall  that  V  (the  first  coordinate  of  T)  behaves  marginally  according 
to  the  same  probability  rules  as  the  original  process  Z.  By  studying  this 
coupled  process  we  can  make  deductions  about  the  asymptotic  properties  of 
the  variance  estimators  of  both  T  and  Q.  The  first  step  in  comparing  the 
asymptotic  properties  of  rp  and  is  the  comparison  of  tiie  point  estimates 
for  E g%\  bp  and  ?q. 
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(1)  THEOREM,  ir  E|ff(960l  <  +oc,  then  limn _ „  ra  -  Ej$'  with 

probability  1. 

Proof.  We  know  that  bp  — *  E<7%'  with  probability  1,  and 


for  any  measurable  function  f.  Since  convergence  to  a  constant  in  probability 
is  the  same  as  convergence  in  distribution,  a  simple  argument  on  the  remainder 
terms  and  then  a  continuous  mapping  argument  implies  that  the  asymptotic 
properties  of  the  estimators  are  the  same.  The  details  of  the  argument  will 
be  omitted.  3 

The  remaining  terms  is  Sq  are  sample  variance  and  covariance  terms.  We 
will  examine  one  of  these  terms,  jjIf  in  some  detail.  The  other  two  can  be 
handled  by  similar  reasoning.  To  study  the  asymptotic  behavior  of  sfj  we 
will  now  define  a  number  of  random  variables  related  to  the  coupled  process 
V>.  Le‘ 

A)(0)  -  flo[T)  -  flb(Q)  -  o 

ft(v,  —  min{  nk  >  (0)  |  Xn*  Mnk  —  1  } 

0,(7*)  —  min{  nk  >  0,_,(7*)  |  Snk  —  1  } 

3,(0.)  —  min{  nk  >  A_j(Q)  |  X„*  —  1  } 

On(rJ/)  —  m«{  i  |  <  n  } 

On(T)  —  max{  »  |  0,(T)  <  n  } 

0n(Q)  -  nux{  .  |  A(Q)  <  n  }. 

The  3  sequences  mark  the  beginning  of  the  cycles  in  their  respective 
sequences,  while  the  U  processes  are  the  counting  processes  for  the  three 


sequences. 


(2)  THEOREM.  If  /  mine<>o  c,  |$(j)|  d*  <  oo,  then 

lim 

n— •  oo 

j  J  | 

""  E{  0A(t)(a) }  r|  (^/  — Ew{A(a)}E.{/(4)»a|  p u  —  a.e. 

for  any  probability  measure  u. 

Note:  (a)  In  this  theorem  we  introduce  the  notion  of  a  supercycle.  A 
supercycle  will  be  a  piece  of  the  coupled  process  ^  between  consecutive  s. 
Observe  that  these  supercycles  are  legitimate  regenerative  cycles,  since  when¬ 
ever  a  supercycle  begins  a  cycle  for  the  T  process  begins  as  well. 

(b)  This  theorem  says  that  the  asymptotic  variance  estimate  equals  the 
ex  pec  ted  sum  of  variances  of  the  quasi-cycles  in  a  supercycle  divided  by  the 
expected  number  of  quasi-cycles  in  a  super  cycle. 

(c)  To  take  advantage  of  the  Strong  I. aw  of  Large  Numbers,  we  must  6nd 
a  way  to  break  the  Q  sequence  iulo  independent  and  identically  distributed 
block,  this  is  one  reason  it  is  convenient  to  couple  the  ^  process  with  the  T 
process. 

(d/  The  notation  that  arises  in  this  theorem  looks  more  intimidating  than 
it  really  is.  First  is  the  number  of  supercycles  completed  by  the  end 

of  the  nlh  quasi-cycle.  Then  denotes  the  end  of  the  last  supercycle 

complete  by  the  end  of  the  n'vh  quasi-cycle.  Finally  Y„  —  Oj ^  is 

the  number  of  quasi-cycles  which  occur  in  those  complete  supercycles.  AI30, 
suppose  that  T'n  -  Q). 

Proof.  Now  has  a  limit  if  and  only  if 

;r~r  IX  -  m  a(q)  >e.<  /(z,i  })•  (3) 


74 


exists.  Furthermore,  when  the  limits  exist  they  will  be  the  same.  To  see  this 


note 

o.(a)  4*<a)-i  n 

/<zj  -  urn  i{  £  £  m  +  £  m  >  mi 

”^'r~  ~  immi  k-i  i(g)  >-fcWQ)  ; 

-E.{/(2i)}  P„-a.e. 

for  any  probability  measure  u,  by  the  ratio  limit  theorem  (Lemma  2.1.7).  Now 


i  n 

“s_=E 


1  a  . 

-  £  M)<  -  £  m 

<  -  mai/fsjut,.  (0)4-1 

n  *€  S 


where  is  the  length  of  the  A;lh  quasi-cycle.  The  last  expression  above  goes 
to  zero  as  n  goes  to  infinity  since  VJt>.(a)-*-l  <  +oo  with  probability  1.  Then 
from  (4)  we  have 


0.(0)  J»(Q)-l 


0.(0)  A(0)-1 


i-sgajE  2  m 

n' '  fc—l  »-J»_i(0) 


-  2s.  oh  JLz  oioj  E  E  „«» 


i  .-4._,(0) 


-M/WJJtagjg 

“  E.{  /(Zo) }  E{  Qoji')+ 1  —  3o.(ft) } 

-E(A(Q)  >E.{/(2o)> 


The  next  to  last  equality  is  true  by  the  discrete  version  of  the  Elementary 
Renewal  Theorem  and  the  last  is  valid  since  the  number  of  quad-cycles  in 
a  supercycle  is  an  i.i.d.  sequence.  Now  we  must  find  the  limit  uf  (3).  This 
limit  can  be  determined  by  breaking  a  sample  path  into  blocks  using  the 
regenerative  points  of  the  coupled  process.  To  do  this  notice 
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J^Tf  XX  -  E.{3,(Q)  }E.{/(Z»)  })> 

J— 1 

i 

-  -  XX  -  E.{  A(Q)  }E.<  /(Zo)  })> 

n 

+  £  (i.,-E.{A(a)>E.{/(Zo)})’. 

*— T.+l 

We  can  show  by  the  usual  arguments  that  the  second  term  goes  to  zero 
as  n  goes  to  infinity  and  we  only  need  to  evaluate  the  first  term.  To  apply  the 
Strong  Law  of  Large  Numbers  let  us  multiply  and  divide  by  Oy„(Qj(0).  There 
are  then  two  limits  to  evaluate.  From  the  Strong  Law  of  Large  Numbers  and 
the  fact  that  -f-oo  as  n  — •  -f-oo,  we  have 

XX  -  K-<  aw  >■=•{/<*) »’ 

t; 

-E  £  (L,-E,{a,(Q|}E.{/(Zo)))5  P„ -<■•«. 

>-T. 

for  any  probability  measure  u. 

The  number  of  quasi-cycles  in  a  supercycle  is  an  i.i.d.  sequence,  so  the 
limits  of  summation  can  be  replaced  by  1  and  We  must  also  evaluate 

lim„— oo  CXj„{<a)(t>)/n.  This  term  represents  the  average  number  of  supercycles 
in  n  quasi-cycles.  This  requires  the  same  counting  process  reasoning  that 
we  used  in  Theorem  4.2.1.  Bound  by  5oJbW(o)(CI)/Ojw(q)(0)  and 

ar,d  use  Strong  Law  and  the  fact  that  /3n(Q)  *-» 
oo  as  n  — »  co  to  inTer  that 

“  E(  }  pv  —  o.c. 
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for  any  probability  measure  u.  This  expectation  is  the  same  for  each  super¬ 
cycle,  so  we  may  choose  k**l.  Therefore  (2)  is  proven.  3 

Now  we  have  demonstrated  that  j®  has  a  limit.  It  is  possible  to  write 
an  analogous  expression  for  merely  by  replacing  Q  with  T  and  L  with  Y 
everywhere.  We  would  like  to  show  that  the  limits  are  these  same.  To  pursue 
this  goal  requires  a  variation  of  Wald's  Lemma. 

(5)  THEOREM.  If  Xi,  Xj,  ...  are  identically  distributed  random  vari¬ 
ables  having  finite  expectations,  and  if  N  is  a  positive  integer-valued  random 
variable  independent  of  the  Xn’s  with  E.V  <  -t-oo,  then 

N 

e£X,-E</V}E{X,}. 

1—1 


Proof.  Let 


Then  we  have 


if  N  >  n, 
if  N  <  n. 


N 


Ex* 


V  V.)  , 

TV-  1 


and 


A/  oo  oo 

xn  -  E  £  XnYn  -  Y* 

no  l  hb|  **«- 1 


(») 


li  XnYn  is  replaced  by  IXnY,,!,  the  interchange  in  (B)  is  justified  because  all 
the  terms  are  positive.  This  implies  that  the  original  interchange  is  allowable 
by  Lebesgue’s  Dominated  Convergence  Theorem.  The  random  variable  Yn  is 
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independent  of  X„,  so  we  obtain 


eE  *«-  Ee(-V.)E{1'.1 

nmm  t  t»»»l 

rv—  i 


oo 

-E{Xv}£P(N>n) 

»— l 

-E{Xi}E  {N}.  I 


We  are  now  ready  to  show  that  the  limits  of  sJ*j  and  s^j  are  the  same. 
Recall  from  Lemma  2.1.5  that  we  can  choose  fc  so  that  the  GSMOP  Z  is 


(O',  w,  X,,fc)-recurrent. 


(7)  THEOREM.  Let  Z  be  an  aperiodic  (O',  v?,  X,  fc)-recurrent  GSMOP 
which  has  invariant  measure  t.  Suppose  that  Z  is  also  (0  ,  X  ,  fc)-iecurrent. 
/!/(*)!<*»  <  °°<  then 


E{<Wa)> 


E< 


V  (LJ-E{J1(Q)}E,{/(Zb)})2 
j-i 


i 

i 


1 


E{oJl(^(r)} 


E< 


V  (V,  —  E{  A(7")  }K.{/(V^,)  J)1. 


Proof.  F  rst  notice  that  6  and  \  are  both  sequences  of  Bernoulli  random 
variables  with  parameter  p  and  the  roles  they  play  in  determining  the 
sequence  are  symmetric;  therefore, 

E{0A(#,(fit)}-E{0A(^(r)}. 
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We  only  need  to  show  the  equivalence  of  the  sums.  Suppose  initially 
that  ip  sm  ir,  the  limiting  distribution  of  Z.  In  this  case  the  terms  in  each  sum 
are  identically  distributed,  the  number  of  terms  in  each  sum  is  independent 
of  each  summand,  aud  all  the  relevant  expectations  are  finite,  so  applying 
Theorem  (5)  in  each  expectation,  we  have 


E< 


Y  (LJ-E{A(Q)}E.{/(Z0)})J 


-  E{  <?*(.,< tt) }  E{  (L,  -  E<  A(0)  }  E,{ /(2b)  })J  ) 


and 

(O^T) 

V  (y,-E(A(r)>E,{/(v0)))J 

-  E{  0A(.,(T)  }E{  (y,  -  E{  (A(T)  }E.{  /(Vb) })’  }. 

As  we  noted  above  E{  O^^O)  }  •"  E{  Ojt^{T)  },  so  we  must  only  show 
the  equivalence  of  the  second  terms.  We  know,  however,  that  E{/3i(T)}  = 
E{  d|(Q)  }  and  E,{/(2b)  }  —  E{/(V'o)  },  so  we  must  only  show  that  E{  Y}  }  =« 
E{  Lj  }  and  E{  T ‘  }  — •  E{  Lj  }.  We  cau  apply  Theorem  (4)  again  to  each  side 
of  the  first  equation  to  see  that 


9\ — I 

E(M  =  EV  /(Vn)-E{A(7*)-l}E.{(Vo)l 
flaiO 


and 


E(/.,)-E  Y  AA)-E{A(Q)-l}E  .{/(Zb)) 

n*0 


The  random  variables  £.(Q)  and  0\(T)  are  both  determined  by  Bernoulli  se¬ 
quences  with  the  same  parameter  and  the  second  terms  are  equal  since  both  \ 


70 


and  Z  are  Markov  chains  with  the  same  initial  distribution,  transition  function 
and  limiting  random  variable. 

Now  we  will  examine  the  second  moments  of  the  cycles  of  T  and  the 
quasi-cycles.  Suppose 


D,(Q) 


if  ft(a)>>. 

if  A(a)  <  J. 


Then  we  can  write 


(  £  c<™  cv(x.)) 

immO  ' 


(f]/(Z.)D.(Q))2. 


Thus 


E< 


*.(<0-1 
V  min  C„/(X,)5  }  -  E 

1  C„>  0  I 


j-0 


(£  f[Zt)D,m 2 
0 


-  £E(/(Z,)£h(a))2 


v— o 


+  2  £  ^{/{ZJDjQmZ^Q)}. 

»»<*»<<*> 


The  interchange  is  justified  if  /(Z.)  is  replace  by  its  absolute  value  throughout, 
since  all  the  terms  would  then  be  positive.  This  implies  that  the  original  in¬ 
terchange  is  allowable  by  Lcbesgue's  Dom.natcd  Convergence  Theorem.  Now 
£)}{ Ql2  ■*»  D^(Q)  and  is  independent  of  V,  for  each  i.  Tliua  we  have 


80 


Similarly  E (Y*)  can  be  decomposed.  Bui  we  Dote  that 

E-{(/(2b))a>-E.{(/(V'o))a  >, 
E{A(r)}-E{A(Q)}, 

E{D„(Q)DJQ)  }  -  E {©..(rjDJD  }, 

E{/(2h)/(4i)}-E{AV'HJ/(»'J}. 

The  Pinal  step  in  the  proof  is  to  establish  that  we  do  not  need  the  initial 
distribution  of  the  chain  to  be  t  in  order  to  claim  that  the  theorem  is  true. 
Recall  that  Theorem  5  demonstrated  that  the  limit#  for  was  true  — a.e. 
for  any  initial  measure  o.  The  terms  in  the  limit  do  not  depend  on  the  initial 
measure.  The  theorem  proof  is  complete.  9 
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Section  4.0.  AN  EXAMPLE 


T re  theory  DEVELOPED  in  Sections  4.3  and  4.4  proposed  a  technique  Tor 
finding  point,  estimates  and  confidence  intervals  for  functions  of  the  equilibrium 
distribution  of  a  GSMO  whose  associated  GSMOP  Z  is  Doeblin  recurrent.  It  is, 
however,  likely  to  be  troublesome  for  a  practitioner  to  determine  the  modified 
densities  required  to  develop  a  truncated  chain  for  Z  if  it  is  Harris  rather 
thau  Doeblin  recurrent.  The  purpose  of  this  section  is  to  test  the  techniques 
proposed  on  a  model  which  is  Harm  recurrent. 

The  model  we  will  examine  is  the  (2,2)-server  cyclic  queue  with  feedback 
and  four  customers.  A  schematic  diagram  of  the  systems  is  presented  in  Figure 
1.  As  we  noted  in  Section  3.1,  there  is  not  a  single  set,  since  at  least  two  servers 
are  active  at  all  times.  The  system  is  aperiodic  and  (.4,^,  X,  fc)-recurrent  for 
some  choices  of  A,  y?,  X  and  k. 

The  first  step  is  to  determine  the  recurrence  parameters  which  are  satisfied 
by  the  system.  In  Section  4.2  we  demonstrated  that  chains  with  densities 
positive  on  [0,  oo)  were  recurrent  with  A  chosen  as  the  subset  of  O'  which  has 
all  its  clocks  less  than  some  U.  If  we  choose  U  sufficiently  large,  the  sample  path 
we  generate  using  the  original  process  should  be  virtually  indistinguishable 
from  the  sample  path  we  would  generate  if  we  truncated  the  densities  at  U.  To 
determine  what  L’  is  appropriate  we  must  know  the  distributions  associated 
with  our  model. 

In  order  to  facilitate  the  calculation  of  theoretical  means  and  variances, 
we  suppose  that  each  server  has  a  gamma  distribution.  Let  both  servers 
at  station  A  have  a  F(2,  1)  distribution;  at  station  B  let  one  server  have  n 
T(2,  1)  distribution,  the  other  a  I'(3,  1)  distribution.  For  these  distributions 
a  relatively  small  choice  of  U  is  sufficient.  IT  L’™20,  for  example,  P(T(2,  1)  > 
10)  -  .9939949  and  P(f(3.  1)  >  10)  —  .9999968. 
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Now  we  must  choose  X  and  k.  Unfortunately,  how  we  choose  these 
parameters  is  largely  a  matter  of  guesswork.  It  is,  of,  course,  possible  to  be 
very  conservative  in  our  choices,  letting  k  —  100  and  X  —  .001,  for  example. 
Such  choices  will  do  little  for  the  efficiency  of  our  simulation,  however. 

Wecan  be  guided  by  a  few  facts,  however.  Our  treatment  of  the  asymptotic 
properties  of  our  variance  estimate  required  that  the  Markov  chain  be  x- 
recurrent  for  our  choice  of  k.  Also  the  ergodic  condition  gives  us  some  help 
for  sets  with  large  ^-measure.  If  supJ  y^|?(x, A)  —  P(y,A)|  <  1  —  e,  then 
Doeblin’s  condition  is  satisfied  with  parameters  (P*(x, -),fc,  1 — i/2,t/2),  which 
means  that  when  P*(x,A)  >  1  — e/2  then  P*(y,A)  >  e/2  for  all  choices  of  y. 

Choosing  k  such  that  Z  is  x-recurrent  for  k  or  determining  appropriated  e 
which  satisfies  the  coefficient  of  ergodicity  for  a  given  k  is  not  straightforward, 
however.  In  order  to  reflect  our  ignorance  about  which  k  and  e  to  chcose.  we 
have  simulated  the  system  with  a  number  of  different  choices  for  each.  We 
have  chosen  k«"10,  20  and  30  with  X—.4,  .0,  .9  and  .99.  We  would  like  to 
choose  X  as  large  as  possible  for  a  given  k  since  this  will  increase  the  number 
of  regenerations  in  a  run  of  fixed  length. 

The  functions  we  choose  to  test  the  method  are  indicators  which  can  be 
used  to  determine  the  equilibrium  distribution  of  system.  As  we  pointed  out 
in  Section  2.2  the  state  space  of  this  system  can  be  described  by  a  pair  to 
indicate  which  servers  at  station  A  are  busy  (xi.xj),  a  pair  to  indicate  which 
servers  at  station  B  are  busy  (xj,x*),  and  a  coordinate  (x^)  to  indicate  how 
many  jobs  are  in  service  at  station  A.  Let  us  suppose  that  there  are  four  jobs 
in  the  system.  The  functions  we  will  test  are  /(x)  »  1{i)(xs}  for  i»l,2,3,4. 
Tables  1  and  2  contain  the  12  combinations  of  X  and  k  for  f\.  Tables  3  and 
4  contain  the  results  for  /j,  5  and  6  contain  /j  and  7  and  8  contain  /«. 

Let  the  feedback  loop  be  rhosen  with  probabi*  ty  .5  at  each  completion 
of  a  service  by  one  of  the  servers  at  station  A.  Under  these  assumptions  the 
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network  can  be  viewed  a,  .  Markov  chain  with  73  state,  and  the  theoretical 
means  and  variances  can  be  calculated. 

In  all  the  table,  that  follow.  P.E.  mean,  the  point  estimate  for  f,  and  Hi.. 

denote,  the  half  lenjth  of  the  conOdence  interval.  Ajaio  we  will  comonte  CO 
percent  confidence  intervals. 
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Before  analyzing  the  results  of  our  simulations,  let  us  consider  the  results 
we  would  expect  to  obtain  and  the  effects  of  varying  X  and  k.  First,  for  our 
functions,  point  estimates  are  obtained  by  dividing  the  portion  of  the  simula¬ 
tion  run  the  function  is  positive  by  the  length  of  the  run.  This  is  the  same 
technique  that  is  used  in  practically  every  method  for  estimating  E{/(S&f)}- 
Therefore  we  would  expect  our  point  estimates  to  be  close  to  the  exact  value, 
regardless  of  the  choice  of  k  and  X. 

The  values  of  k  and  X  should  effect  the  variance  calculation,  however. 
La  specifying  X  and  k,  we  are  hypothesizing  that 

P*(xM)>X^)  (lj 

for  all  x  and  measurable  A.  If  we  choose  yj  =■  *,  for  example,  the  geometric 
convergence  of  iterates  of  P  to  x  (see  Lemma  2.1.17)  leads  us  to  believe  that 
larger  values  of  k  admit  more  values  of  X  which  satisfy  (1).  This  implies  that 
the  larger  the  value  of  k,  the  more  values  of  X  will  yield  acceptable  estimates 
of  a.  On  the  other  hand,  smaller  choices  of  X  should  also  yield  more  accurate 
estimates  of  a  since  smaller  values  of  k  would  satisfy  (1)  for  a  given  X.  Both 
effects  (choosing  smaller  X  and  larger  k)  tend  to  lengthen  each  regenerative 
cycle  and  thus  lengthen  the  ‘real’  time  a  simulation  of  a  fixed  number  of  cycles 
requires. 

Now  let  us  turn  to  the  results  of  the  simulation.  As  we  expected,  the 
coverage  of  the  true  value  of  the  estimated  parameter  was  good,  and  was 
fairly  uniform  over  various  choices  of  X  and  k.  The  true  value  of  E{  f%  )  v/as 
covered  in  1 1  of  12  runs  for  X  .4,  .9  and  .99  and  in  10  of  12  runs  for  X  <■=  .6. 
Overall,  the  true  value  was  covered  in  92  percent  of  the  runs.  The  coverage 
was  also  fairly  uniform  over  k.  For  k  «  10,  there  was  coverage  in  13  of  16 
runs;  for  k  — »  20,  14  of  10  runs,  and  for  k  «=  30,  10  of  10  runs. 

Now  let  us  consider  the  variance  estimates  the  simulations  obtained.  As 
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we  expected,  the  larger  the  value  of  k  the  better  the  estimate.  The  improve¬ 
ment  is  particularly  dramatic  for  k  *=>  30.  Of  the  10  runs  with  k  =*  30,  13 
estimated  the  variance  constant  with  an  error  of  less  than  5  percent.  For 
k  »  20,  7  of  10  runs  had  an  error  of  less  than  5  percent,  0  mor  had  an 
error  between  5  and  10  percent.  For  k  =—  10,  5  and  7  were  the  corresponding 
numbers.  This  data  seems  to  confirm  our  intuition  about  the  effect  of  k. 

On  the  other  hand,  varying  the  choice  of  X  produced  puzzliog  results. 
For  X  *■  .6,  .9  and  .99,  the  results  tend  to  confirm  our  intuitions.  For  the 
runs  for  X  *=  .4,  unexpected  results  occurred.  Of  the  9  runs  (of  43  total)  in 
which  the  variance  estimated  differed  from  the  true  value  of  a  by  more  than 
10  percent,  5  occurred  with  X  =*•  .4.  We  are  considering  a  small  sample,  but 
nevertheless,  this  is  somewhat  surprising. 

Finally,  it  is  interesting  to  note  that  the  results  were  good  for  k  *  10 
for  large  values  of  X.  We  would  not  expect  that  P10(/i)  >  .99ir(.4)f  yet  the 
results  for  this  choice  of  parameters  was  very  good. 
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CHAPTER  V 


CONCLUSIONS 


At  the  use  of  stochastic  systems  becomes  more  extensive  in  the  study 
of  complex  phenomena,  the  need  for  a  theoretical  understanding  of  these 
models  and  a  practical  means  of  analyzing  them  becomes  more  acute.  We  have 
proposed  an  approach  which  demonstrates  the  inherent  regenerative  structure 
of  many  of  these  stochastic  systems  and  capitalizes  on  that  structure  in  a 
way  that  allows  us  to  estimate  many  quantities  conerned  with  the  behavior 
of  the  system  in  the  ‘long  run'.  Our  task  now  to  reexatnin<*  the  approach  and 
evaluate  its  strengths  and  weaknesses. 

The  major  weaknesses  of  the  approach  are  obvious.  Certainly  the  av 
sumptions  we  have  made  about  the  system  under  study  have  been  stricter 
than  we  would  like.  The  most  important  of  these  are  the  requirement  that  S  be 
finite  and  the  assumption  that  a  GSMO  without  a  single  set  must  be  Doehlin 
recurrent  in  order  to  apply  the  estimation  procedure  in  Sections  4.1  nod  4.5. 
S  may  be  infinite  (in  an  open  network,  for  example)  and  the  densities  of  the 
clocks  may  have  support  on  the  entire  positive  half  line  in  many  situations 
of  interest.  In  practice,  these  difficulties  may  be  circumvented  by  restricting 
S  and  truncating  the  densities  in  such  a  way  that  the  system  spends  only  a 
tiny  portion  of  time  outside  this  set.  Also  the  example  in  Section  4.6  seems 
to  indicate  that  it  is  not  necessary  to  explicitly  truncate  the  distributions. 


Nevertheless,  it  is  theoretically  preferable  not  to  require  these  approximation 
procedures. 

The  chief  practical  difficulty  lies  in  choosing  X  and  k.  At  present  the 
choice  is  largely  guesswork.  It  is  interesting,  however,  that  many  choices  seem 
to  give  good  results  — even  when  we  would  guess  that  the  parameters  do  not 
satisfy  the  recurrence  condition. 

The  strengths  of  our  approach  are  equally  clear.  By  demonstrating  that 
GSMOPs  are  frequently  recurrent,  we  can  use  the  Athreya-Ney-Nummelin 
construction  to  find  regenerative  processes  that  are  closely  related  to  the 
original  GSMQP.  This  allows  us  to  say,  at  least  theoretically,  a  great  deal  about 
the  process  of  interest.  Furthermore,  with  only  a  small  amount  of  additional 
effort,  we  can  find  point  estimates  and  confidence  intervals  for  functions  of  the 
process  when  it  is  equilibrium,  even  when  we  cannot  explicitly  determine  a 
sample  path  for  a  companion  regenerative  process.  The  procedures  we  propose 
seem  to  be  applicable  to  a  large  variety  of  stochastic  systems,  including  many 
queueing  networks. 

A  study  of  this  kind  can  never  answer  all  questions  that  ari'e  from  the 
development  of  a  technique  and  we  will  now  consider  a  few  that  remain  un¬ 
answered  by  our  discussion.  First  we  would  like  to  determine  which  of  our 
assumptions  can  be  relaxed.  The  three  extensions  we  would  like  to  make 
are:  (1)  allowing  S  to  be  countably  infinite — this  generalization  might  require 
a  good  deal  of  extra  work,  we  frequently  used  the  fact  that  S  was  finite; 
(2)  allowing  speeds  other  than  1  in  GS.MOs  without  single  sets — this  would 
probably  lead  to  a  notational  nightmare;  and  (3)  requiring  a  GSMO  to  be 
only  Harris  recurrent  in  order  to  use  our  independent  sequences  procedure — 
this  generalization  might  be  obtainable. 

Second,  in  order  to  make  the  method  accessible  to  the  practitioner,  we 
must  improve  our  means  of  choosing  X  and  k.  Although  the  procedure  seems 
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fairly  robust,  it  is  clearly  unsatisfactory  to  have  only  one’s  intuition  about 
the  network  to  guide  the  selection  of  these  parameters. 

Finally,  this  technique  is  designed  to  provide  ways  of  estimating  func¬ 
tions  of  the  equilibrium  distribution  of  a  GSMP.  In  the  simulation  of  queueing 
networks,  for  example,  it  is  often  desireable  to  make  inferences  about  the 
response  time  (the  time  required  for  a  job  to  traverse  some  portion  of  the 
network).  This  is  usually  considered  to  be  a  separate  question  but  it  would  be 
interesting  to  determine  if  the  companion  regenerative  processes  to  a  network 
could  be  used  to  extend  to  a  more  general  setting  the  existing  results  in  this 
area. 
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system  can  be  made  Markovian.  For  CSMPs  the  supplementary  variables  are 
clocks  which  record  the  amount  of  time  until  the  occurrence  of  various  events 
which  could  influence  the  system.  In  a  queueing  network,  for  example,  each 
server  and  each  arrival  stream  would  be  associated  with  a  clock.  By  Including 
these  clock  readings  as  part  of  the  description  of  the  system,  only  the 
present  state  of  the  system  is  required  to  predict  future  behaviour.  This 
means  that  the  new  model  Is  Markovian  and  therefore  amenable  to  analysis  via 
the  use  of  Markov  chain  theory.  To  us*  these  processes  for  simulation  pur¬ 
poses  a  central  limit  theorem  Is  required.  Obtaining  this  result  Is  the 
second  goal  of  this  paper.  4  Our  approach  to  this  problem  Is  to  find  closely 
related  regenerative  processes  on  which  to  base  the  central  limit  theorem  for 
the  process  under  study.  New  results  in  the  theory  of  Markov  chains  on  a 
general  state  space  make  It  clear  how  these  regenerative  processes  can  be 
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